Description of problem: I first noticed this problem on CentOS 6 while studying for the RHCSA exam and decided to try it out on my Fedora 20 box to see if it was still an issue and it is. When you have two virtual networks created and you create a VM using each network, only one VM can ping or connect to the other. One system pings fine and the other machine gets an error of "Destination Port Unreachable". I took a look at the firwall rules on the host machine and see something like this. Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination 1 35 2684 ACCEPT all -- * virbr1 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED 2 35 2684 ACCEPT all -- virbr1 * 192.168.122.0/24 0.0.0.0/0 3 0 0 ACCEPT all -- virbr1 virbr1 0.0.0.0/0 0.0.0.0/0 4 8 480 REJECT all -- * virbr1 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 5 0 0 REJECT all -- virbr1 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 6 17699 3387K ACCEPT all -- * virbr0 0.0.0.0/0 192.168.100.0/24 ctstate RELATED,ESTABLISHED 7 32439 40M ACCEPT all -- virbr0 * 192.168.100.0/24 0.0.0.0/0 8 0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0 9 0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 10 0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable I'm assuming these rules are added by libvirt and from these rules we can see that any traffic leaving virbr1 going to virbr0 will be accepted by rule #2; however, traffic leaving virbr0 going to virbr1 will be rejected by rule #4. If rule #4+5 are moved below #8 then everything works fine or if you move rule #7 below rule #3 it also works. I am not sure whether the correct behavior is to not have the two networks able to see each other by default or that they should both see each by default, but I assume that working one way but not the other is not the desired behavior. Version-Release number of selected component (if applicable): libvirt 1.1.3.3-5 How reproducible: 100% Steps to Reproduce: 1. Create a second network using the Virtual Machine manager 2. Create two VMs, one on the default network and another on the new network. 3. Bring up both machines and try to ping between them. Actual results: Ping only works one way, which is the machine connected to the interface listed first in the FORWARD chain of the filter table. Expected results: Either both machines should be able to ping each other or neither machine should be able to ping the other.
This problem has existed for a *very* long time. It was first reported in Bug 453580. So far it hasn't been important enough to anyone to fix it, and so things have remained as they are for the last 5 years. The *desired* behavior is that guests on different isolated and NATed networks should *not* be able to communicate with each other. The problem is that it's not possible to do that by merely re-ordering the rules, even if all the rules for all networks were added at the same time (and thus could be arbitrarily re-ordered) which anyway isn't the case (each network can and is started/stopped independently of all other networks). The presence of the nat table rules, which are apparently handled prior to the FORWARD rules, makes it even more complicated. Lacking any solution from someone more intelligent about iptables, the only way I can see to make this work would be to add a specific REJECT rule for "-i virbrX -o virbrY" for every pair of NATed/isolated networks, which is not at all scalable. Fortunately, in the time since this was first reported, it hasn't been an important issue for anyone. But of course as always, that is subject to change, and patches are always welcome.
(In reply to Laine Stump from comment #1) Ok, thanks for the reply. I tried to search for this problem before I posted it, but I guess I didn't do a good job of that. I have a workaround for it at the moment so it's not big deal and I don't see myself trying to use this in any kind of production environment at the moment. It was just something that came up in the book I was using to study for the RHCSA exam.
*** This bug has been marked as a duplicate of bug 453580 ***
FYI we believe we've finally solved the problem with this series https://www.redhat.com/archives/libvir-list/2018-November/msg00018.html