Description of problem: When I run a VM on a bridged Network (Bridge0, connected to enp39s0), the VM cannot get a DHCP4 address only IPv6. Version-Release number of selected component (if applicable): Fedora 32 (it worked on Fedora 31) How reproducible: Create Bridged Network, add new VM on Bridged Network, try to boot via PXE Steps to Reproduce: 1. Create new VM on Bridged Network 2. Start VM. No PXE Boot. 3. Boot from CD, enable Network: VM only get's an IPv6 address Actual results: No PXE Boot, no IPV4 address through DHCP Expected results: PXE Boot Menu and valid IPv4 address Additional info: Setup worked flawless on Fedora 31, But showed up after Update to Fedora 32 No user rules on the firewall
Created attachment 1686083 [details] Screenshot Virt-Viewer, Start VM with firewall Default Launching a CentOS VM on a Bridged Network with the Default FW-Settings on the Fedora 32 Host. VM only gets an IPv6 adress.
Created attachment 1686084 [details] Virt-Viewer. VM start with firewalld turned off on F32 host After turning off FirewallD on Fedora 32 Host, the bridged VM gets an IPv4 and IPv6 adress as expected. -> On Fedora 31 the VM worked this way with FirewallD switched on.
I have similar issue. I have multiple VMs that have worked for years in this setup that now I have to disable firewalld for them to get IPv4 addresses.
Nathan and Andreas - if you manually set and IPv4 address, does IPv4 work? (i.e. is it just DHCP that's being blocked? Or is it all IPv4 traffic? Eric - has there been a change to firewalld rules in F32 vs F31 you're aware of that would break IPv4 for a tap device connected to a bridge device that's attached to the host's ethernet?
Wait. firewalld switched from using the iptables backend in F31 to using the nftables backend in F32. However, RHEL8 firewalld is also using the nftables backend, and traffic goes across ethernet-attached bridges with no problems there. Eric? I'm re-assigning this to firewalld, since it's occuring with VMs attached to a bridge that is directly attached to an ethernet (not to a libvirt-created bridge). libvirt has never done any sort of management of the firewall for bridges like this, which are created by the host network config (usually NetworkManager) - traffic between VMs attached to these bridges and the physical network has always been untouched by the host system firewall.
Oh piffle. It's because something is again causing the br_netfilter module to load, and so all traffic traversing bridges is sent through the firewall. This happens no matter what the firewalld backend is set to. It isn't a firewalld problem after all. We just need to figure out who is causing br_netfilter to load. There was a problem with this a year or two ago, where attempting to *remove* a particular type of iptables rule would cause iptables to autoload br_netfilter (or something like that). I'll try a few things out in the morning. But in the meantime I'm moving the bug back to libvirt (it's not our bug, but it *really* isn't firewalld's bug :-) In the meantime, if you run "rmmod br_netfilter", your guests will once again be able to pass IPv4 traffic (until whatever is autoloading it does that again, anyway).
Same here. After the upgrade to F32, my bridged VMs are not getting IP with DHCPv4. I'm sniffing at the external switch and the broadcast packets never reach out of the host. kernel-5.6.10-300.fc32.x86_64 libvirt-daemon-6.1.0-2.fc32.x86_64 firewalld-0.8.2-2.fc32.noarch
Sigh. On a freshly installed F32 Workstation: # cat /etc/modules-load.d/podman.conf br_netfilter <====== this means that br_netfilter will be modloaded at every boot. dnf provides /etc/modules-load.d/podman.conf [...] podman-2:1.9.1-1.fc32.x86_64 : Manage Pods, Containers and Container Images Repo : @System Matched from: Filename : /etc/modules-load.d/podman.conf On an F31 machine: (/etc/modules-load.d/podman.conf doesn't exist) # rpm -qa | grep podman podman-1.9.0-1.fc31.x86_64 podman-plugins-1.9.0-1.fc31.x86_64 # dnf provides /etc/modules-load.d/podman.conf [...] Error: no matches found So between podman-1.9.0 and 1.9.1 podman has apparently added default configuration that forces br_netfilter to be loaded unconditionally at boot time, and since the default for net.bridge.bridge-nf-call-iptables (which takes effect whenever br_netfilter loaded) is 1, this means that all traffic traversing the bridge will be sent through iptables/nftables (and most likely fail if there are any sort of rules there at all). IMO, podman shouldn't be assuming they can change such a basic behavior of networking as the default - if they need this filtering (which apparently is because they want to take advantage of port redirection DNAT rules? - see Bug 1703261) then they should at least make it configurable, and OFF by default. I'm moving this to podman to see what they have to say about it.
Apologies I just saw this now. Although it seems the source has been found, after ipv4 address is assigned by temporarily disabling firewalld normal IPv4 traffic flows as expected. If the br_netfilter module is the culprit and podman needs it what is my work around? I did install podman on this machine to start playing with it but there are no containers running when the problem occurs.
Unloading br_netfilter solves the problem. Disabling firewalld while leaving br_netfilter loaded does *not* work. IPv4 DHCP gets through then, but TFTP is blocked so PXE-booting VMs can't load pxelinux.0.
Ah yeah sorry Andreas, in my case I'm not using PXE/TFTP. Just DHCP. In my case unloading it and booting a VM allows DHCP through with firewalld enabled. With br_netfilter loaded I can get DHCP through by disabling firewalld for DHCP and then re-enabling it for normal traffic. Looks like different use cases.
Sorry about this one. I've talked with the rest of the Podman team, and we're going to revert this. We were unaware there would be an impact on Libvirt - wouldn't have landed this if we realized.
This is most likely related to https://bugzilla.redhat.com/show_bug.cgi?id=1832723. The same upstream fix should address this BZ and that one, but with different packaging needs.
My last comment pointed to this BZ rather than the correct one: https://bugzilla.redhat.com/show_bug.cgi?id=1703261
When you say "revert", you mean that you'll revert it upstream and also backport the revert to Fedora 32 and Rawhide, correct? As long as this remains in the fedora packages, we will continue to get bug reports that bridged networking no longer works. Thanks!
Podman v1.9.2, which should be in updates-testing now for F31/F32, removes the `br_netfilter` requirement.
Assigning to Jindrich for any packaging needs.
podman-1.9.3-1.fc32.x86_64 - current version for Fedora 32 - still contains /etc/modules-load.d/podman.conf which loads br_netfilter. I confirm that unloading br_netfilter solves the problem with my virtual machines using a bridge connected to an ethernet device. Another solution would be to disable the use *tables when br_netfilter is loaded. It seams that the br_netfilter adds the following kernel parameters, they don't exist when br_netfilter is not loaded. # sysctl -a|grep bridge net.bridge.bridge-nf-call-arptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-filter-pppoe-tagged = 0 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-pass-vlan-input-dev = 0 Setting net.bridge.bridge-nf-call-iptables=0 also solves the problem for IPv4 for me. net.bridge.bridge-nf-call-ip6tables=0 will do the same for IPv6. I'm not sure if it neccessary to set net.bridge.bridge-nf-call-arptables=0. For being permanent, these assignments can be put in a file in /etc/sysctl.d/*.conf (as a temporary workaround for admins). I have tried to set some rules for firewalld or nft when br_netfilter was loaded and the bridge kernel parameter on default value, but I was not successful. I think it would be more flexible to have firewall rules that allow or deny forwarding over a bridge in general and selectively for bridges and slave devices (e.g. virtual machines). So some vms and containers can filer there network access, and others may block or allow all. It would also be fine to have options / parameters for libvirt / virsh / virt-manager to set network filter rules on network devices assigned to virtual machines that are automatically applied when the vm starts (because it created a dynamic network device vnet*). I found nwfilter* command in virsh, but no documentation and examples how I can assign rules to a vm. virt-manager doesn't show any configuration options about nwfilter.
Created attachment 1695942 [details] File to save as /etc/sysctl.d/81-bridge-nf-off.conf as a temporary workaround until we have a final solution
Fixed in podman-2.0.0-rc4.1.fc32
(In reply to Daniel Walsh from comment #21) > Fixed in podman-2.0.0-rc4.1.fc32 Thanks for the update. But is podman 2.0.0 rc4 ready for production? Bodhi: "This update's test gating status has been changed to 'failed'." Status is still Pending->Testing. Brent Baude has build podman 2.0.0 rc5 on koji in the meantime. If podman 2.* isn't ready for general use now, I suggest build an updated version of podman 1.9.* which is currently shipped with Fedora 32.
We are working to get podman 2.0 out the door. It will be the next release.
Closing, please reopen if issue isn't fixed.