Fedora Account System
Red Hat Associate
Red Hat Customer
1. Describe the problem: A Windows 11 VM running under libvirt/QEMU/KVM (default NAT network, virbr0, 192.168.122.0/24) has no internet access when the host is connected via Wi-Fi (Intel Wi-Fi 6E AX211, iwlwifi driver), but works perfectly when the host is connected via Ethernet (USB-C adapter, cdc_ncm driver) instead — same VM, same libvirt config, only the host's active network interface differs. Outbound traffic from the VM (TCP SYN, ICMP echo-request) is correctly routed, masqueraded (SNAT) and reaches the internet. Reply packets (TCP SYN-ACK, ICMP echo-reply) arrive correctly on wlp0s20f3, are recognized by conntrack (ct direction reply, snat-done, dnat-done) and have their destination correctly rewritten back to the VM's IP (192.168.122.x) as early as nftables' PREROUTING hook. The connection then gets stuck forever in conntrack state SYN_RECV (TCP) and the packet never reaches the netfilter FORWARD hook, the virbr0 bridge, or the VM's vnet/tap interface — it is silently lost, with no netfilter drop verdict, no interface error/drop counter incremented anywhere (checked ip -s link, /proc/net/netstat, nstat), no SELinux AVC denial, and no rp_filter/martian log entry. Using bpftrace on tracepoint:skb:kfree_skb during a live reproduction, the exact drop point was identified: reason=SKB_DROP_REASON_IP_INHDR loc=ip_forward This fires consistently and repeatedly (dozens of times per test) for every reply packet destined to the VM. SKB_DROP_REASON_IP_INHDR corresponds to the explicit software IP header checksum verification (ip_fast_csum) performed at the very start of net/ipv4/ip_forward.c's ip_forward(), BEFORE the packet ever reaches the netfilter NF_INET_FORWARD hook — which is exactly why nft monitor trace never shows these packets past PREROUTING's accept verdict, and why no standard netfilter-level instrumentation can see the drop. Suspected mechanism: the AX211/iwlwifi hardware always performs RX checksum offload (ethtool reports rx-checksumming: on [fixed] — not togglable, hardware-locked), so incoming packets are marked skb->ip_summed = CHECKSUM_UNNECESSARY. When conntrack/nf_nat rewrites the destination address during de-NAT in PREROUTING, the incremental IP header checksum update does not appear to correctly update the on-wire checksum bytes for packets in this ip_summed state — or leaves them in a state that ip_forward()'s unconditional software re-verification then fails. The Ethernet adapter (cdc_ncm) has rx-checksumming: off [fixed], meaning all packets are always fully software-verified/computed, which is likely why the same NAT setup works flawlessly over that interface. 2. What is the Version-Release number of the kernel: kernel-core-7.1.8-200.fc44.x86_64 (uname -r: 7.1.8-200.fc44.x86_64) 3. Did it work previously in Fedora? If so, what kernel version did the issue *first* appear: Unknown — this is a fresh Fedora 44 install; no earlier kernel was available to bisect against (only this kernel and an identical rescue entry are installed). 4. Can you reproduce this issue? If so, please provide the steps to reproduce: - Host: Fedora 44, kernel 7.1.8-200.fc44.x86_64, Dell Latitude 7340 (Intel Core i5-1345U), Intel Wi-Fi 6E AX211 (iwlwifi). - Connect the host to a Wi-Fi network (any AP). - Create/start a libvirt VM (tested with Windows 11 + VirtIO drivers, and reproduced on a second, freshly-created VM) attached to libvirt's default NAT network (virbr0). - From inside the VM, run `ping 8.8.8.8` or open any web page. - Outbound requests leave fine; no replies are ever received in the VM. Connection stays in conntrack state SYN_RECV / never completes. - Reconnecting the host via Ethernet instead of Wi-Fi (same VM, same libvirt config, no other change) immediately fixes it. Troubleshooting already performed without success (all ruled out as the cause): - firewalld zone/policy configuration (masquerade, inter-zone forward policy, zone forward=yes) — verified correct via nft monitor trace. - rp_filter set to 0 on all relevant interfaces — no change. - Hardware offload toggles via ethtool: rx/tx-checksumming, GRO, GSO, TSO individually disabled on wlp0s20f3 and/or virbr0 — no change. - A tc filter with `action csum ip tcp icmp udp` added to virbr0's egress qdisc (mirroring libvirt's own DHCP checksum-fix mechanism) — no change (this hook runs after ip_forward() already dropped the packet). - A tc `clsact` ingress filter with `action csum ip continue` added directly on wlp0s20f3 — no change (this hook runs before NAT rewrites the destination, so it can't fix the checksum for a rewrite that hasn't happened yet). - WiFi power save disabled (iw dev wlp0s20f3 set power_save off) — no change. - Both libvirt firewall backends tested (native nftables AND iptables/iptables-nft) — identical symptom under both. - Full host reboot — no change. - br_netfilter module not loaded (not applicable). SELinux Enforcing, no AVC denials. MTU consistent (1500) everywhere. ARP/neighbor resolution and bridge FDB/STP state (forwarding) all correct. Happy to provide the full bpftrace output, nft monitor trace logs, or test further patches/module parameters on request. Reproducible: Always
Created attachment 2154653 [details] bpftrace kfree_skb trace: exact kernel drop point (SKB_DROP_REASON_IP_INHDR in ip_forward)
Created attachment 2154654 [details] tcpdump on VM tap interface: outbound requests only, replies never arrive at the VM
The bpftrace result narrows this further than the checksum theory: in 7.1.8, ip_forward() sets SKB_DROP_REASON_IP_INHDR in exactly one place, the too_many_hops label (net/ipv4/ip_forward.c:118, 173-177), reached when ip_hdr(skb)->ttl <= 1. ip_forward() does not verify the IP checksum at all; that check runs earlier, in ip_rcv_core() (net/ipv4/ip_input.c:554), and fails with a different drop reason (IP_CSUM). Your replies already passed it -- you saw them de-NATed in PREROUTING. So the replies are arriving on wlp0s20f3 with TTL 1. Locally-terminated traffic never checks TTL, which is why the host itself works on Wi-Fi; forwarding does, which is why only the VM path dies; Ethernet is a different first hop. TTL clamping is a known anti-tethering behaviour of phone hotspots and some AP/CPE firmware. To confirm, ping 8.8.8.8 from the VM while running sudo tcpdump -ni wlp0s20f3 -v icmp and check the ttl of the incoming echo replies. The same capture should also show the ICMP time-exceeded the host emits for each dropped reply (ip_forward.c:176). If it shows ttl 1, the AP is rewriting it and the kernel is behaving as specified. Workaround until the AP is fixed -- restore the TTL on ingress, before routing: nft add table ip raw; nft add chain ip raw pre '{ type filter hook prerouting priority raw; }'; nft add rule ip raw pre iifname "wlp0s20f3" ip ttl lt 2 ip ttl set 64
Hi, Thank you for the diagnosis — the nftables workaround works perfectly. I tested it with a Windows 11 VM (libvirt/QEMU) and it now has full internet access over Wi-Fi, exactly as it does over Ethernet. I also did a full audit of my local network to make sure the TTL clamping isn't coming from any of my own equipment, since you mentioned it's typically caused by an access point doing anti-tethering: ONT (Nokia G-010G-Q): pure Layer 2 bridge per the manufacturer datasheet, no NAT/routing capability at all. AP (TP-Link EAP660HD): pure access point, no NAT capability, and TP-Link/Omada documentation confirms no TTL-related or anti-tethering feature exists on this product. Router (TP-Link Omada ER605 v2): the only NAT device in my chain, fully under my control — checked the complete user guide, no TTL manipulation setting exists. My full setup is: FTTH fiber (OVH Telecom, over Bouygues Télécom's collection network) → ONT → ER605 → switch → EAP660HD, entirely Gigabit, no ISP-provided router/CPE anywhere in the chain. Since none of my local equipment is capable of this, the TTL=1 must be applied upstream on the ISP's network (OVH Telecom / Bouygues collection infrastructure). I've opened a support ticket with OVH Telecom to ask them to confirm and hopefully disable this at the source. Given this appears to be caused by the ISP's network rather than a Fedora/kernel issue, feel free to close this bug or reclassify it as needed. I just wanted to confirm the workaround works and share the root-cause investigation in case it helps someone else with a similar libvirt-behind-a-clamped-WAN setup find this bug and the fix. I'll post an update here if OVH confirms/resolves it on their end. Thanks again for the help.
Nothing here needs a kernel change, so this can be closed as NOTABUG whenever suits you or the maintainers -- the TTL rewrite happens upstream of your equipment and the nft rule is the fix. One thing did come out of it. SKB_DROP_REASON_IP_INHDR, the reason you traced, is also what the kernel reports for a genuinely malformed IP header, which is why reading it as a checksum failure was reasonable. A patch adding a separate SKB_DROP_REASON_IP_TTL_EXCEEDED is now on netdev, so the next person tracing this sees the TTL straight away: https://lore.kernel.org/all/20260825073906.336072-1-junjie.cao@intel.com/ It changes no behaviour: the packets still drop, and the nft rule is still what you need. Still interested in what OVH come back with.