Bug 622649 - Guest can not get ip adress because libvirt iptables rules were not added
Guest can not get ip adress because libvirt iptables rules were not added
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.0
x86_64 Linux
low Severity high
: rc
: 7.0
Assigned To: Peter Krempa
Virtualization Bugs
: TestOnly
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-08-09 23:40 EDT by yanbing du
Modified: 2014-06-13 07:34 EDT (History)
13 users (show)

See Also:
Fixed In Version: libvirt-1.1.1-1.el7
Doc Type: Bug Fix
Doc Text:
Libvirt uses transient iptables rules for managing NAT or bridging to virtual machine guests. Any external command that reloads iptables state ( such as running system-config-firewall) will overwrite the entries needed by libvirt. Consequently, after running any command or tool that changes the state of iptables, guests may lose access the network. To work around this issue, use the command 'service libvirtd reload' to restore libvirt's additional iptables rules.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 07:34:11 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description yanbing du 2010-08-09 23:40:30 EDT
Description of problem:
  The guests can not get ip address. And there is no libvirt related rules in iptables.
# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-is-bridged 
2    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         



Version-Release number of selected component (if applicable):
build-RHEL6.0-20100807.0
kernel-2.6.32-59.1.el6.x86_64 
libvirt-0.8.1-21.el6.x86_64
qemu-kvm-0.12.1.2-2.108.el6.x86_64
qemu-img-0.12.1.2-2.108.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Install the host via CDROM.
  
Actual results:
libvirt iptabls rules were not added.
After install a guest via CDROM, this problem lead to the guest can not get ip adress.

Expected results:
Guest can get ip adress,libvirt iptabls rules were added.

Additional info:
#virsh net-destroy default
#virsh net-start default
# service iptables status

Table: nat
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
2    MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
3    MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24
Comment 2 RHEL Product and Program Management 2010-08-09 23:58:25 EDT
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 3 Eric Blake 2010-08-10 00:04:33 EDT
I'm wondering if this is related to bug 617505.  I'm also seeing some weird behavior where system-config-firewall hangs trying to update the iptables rules, or where 'iptables -L' shows fewer rules than what I know I set up in system-config-firewall.  In other words, this may be a kernel or iptables bug, rather than a libvirt bug.
Comment 5 Daniel Berrange 2010-08-10 07:15:48 EDT
FYI, any time that system-config-firewall is run, it will blow away all the iptables rules created by libvirt. You need to run 'service libvirt reload' to re-create them.
Comment 7 Lawrence Lim 2010-08-11 21:02:10 EDT
(In reply to comment #5)
> FYI, any time that system-config-firewall is run, it will blow away all the
> iptables rules created by libvirt. You need to run 'service libvirt reload' to
> re-create them.    

Please re-test based on Daniel's suggestion.
Comment 8 yanbing du 2010-08-11 22:32:37 EDT
On the new host, before do any action, check the iptables, as following:

[root@dhcp duyanbing]# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-is-bridged 
2    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

[root@dhcp duyanbing]# service libvirtd reload
Reloading libvirtd configuration:                          [  OK  ]
[root@dhcp duyanbing]# service iptables status
Table: nat
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
2    MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
3    MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Table: mangle
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    CHECKSUM   udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:68 CHECKSUM fill 

Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53 
2    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
3    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:67 
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
5    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
6    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
7    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
8    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
9    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED 
2    ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
6    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-is-bridged 
7    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

[root@dhcp duyanbing]# system-config-firewall

[root@dhcp duyanbing]# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:21 
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
6    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-is-bridged 
2    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Result:just like Daniel's comment, everytime after run system-config-firewall, need run 'service libvirt reload' to re-create libvirt iptables rules.
Comment 10 yanbing du 2010-08-18 01:38:57 EDT
As default, after install a new host with build-RHEL6-20100811.2, the iptables have the libvirtd related rules.
# service iptables status
Table: mangle
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    CHECKSUM   udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:68 CHECKSUM fill 

Table: nat
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
2    MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
3    MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         
...
But, after system-config-firewall is run, it will blow away all the
iptables rules created by libvirt.
# system-config-firewall

# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:21 
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
6    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-is-bridged 
2    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination
Comment 14 Denise Dumas 2010-09-10 15:56:42 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Libvirt uses transient iptables rules for managing NAT or bridging to virtual
machine guests.  Any external command that reloads iptables state from scratch,
such as running system-config-firewall, will overwrite the entries needed by
libvirt.

> * Consequence
>     What happens when the bug presents.

Guest VMs can no longer access the network, because the traffic is being
dropped by iptables rules that are too strict.

> * Workaround
>     What actions, if any, reduce the bug’s effects or prevent it from
> presenting 
>     entirely.

After running any command that might change iptables state, such as
system-config-firewall, follow up with the command 'service libvirt reload' to
restore libvirt's additional iptables rules.

> * Result
>     If the bug’s effect is only reduced, what consequences still flow;
>     if the bug is prevented, what restrictions does the workaround impose.

Should be no restrictions - anyone with enough privileges to run
system-config-firewall also has enough privileges to run service libvirt
reload.
Comment 17 Ryan Lerch 2010-09-28 22:00:06 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,27 +1 @@
-Libvirt uses transient iptables rules for managing NAT or bridging to virtual
+Libvirt uses transient iptables rules for managing NAT or bridging to virtual machine guests. Any external command that reloads iptables state ( such as running system-config-firewall) will overwrite the entries needed by libvirt. Consequently, after running any command or tool that changes the state of iptables, guests may lose access the network. To work around this issue, use the command 'service libvirt reload' to restore libvirt's additional iptables rules.-machine guests.  Any external command that reloads iptables state from scratch,
-such as running system-config-firewall, will overwrite the entries needed by
-libvirt.
-
-> * Consequence
->     What happens when the bug presents.
-
-Guest VMs can no longer access the network, because the traffic is being
-dropped by iptables rules that are too strict.
-
-> * Workaround
->     What actions, if any, reduce the bug’s effects or prevent it from
-> presenting 
->     entirely.
-
-After running any command that might change iptables state, such as
-system-config-firewall, follow up with the command 'service libvirt reload' to
-restore libvirt's additional iptables rules.
-
-> * Result
->     If the bug’s effect is only reduced, what consequences still flow;
->     if the bug is prevented, what restrictions does the workaround impose.
-
-Should be no restrictions - anyone with enough privileges to run
-system-config-firewall also has enough privileges to run service libvirt
-reload.
Comment 18 Laine Stump 2010-11-16 16:26:52 EST
Just noticed a typo in the recommended workaround for the problem...
Comment 19 Laine Stump 2010-11-16 16:26:52 EST
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Libvirt uses transient iptables rules for managing NAT or bridging to virtual machine guests. Any external command that reloads iptables state ( such as running system-config-firewall) will overwrite the entries needed by libvirt. Consequently, after running any command or tool that changes the state of iptables, guests may lose access the network. To work around this issue, use the command 'service libvirt reload' to restore libvirt's additional iptables rules.+Libvirt uses transient iptables rules for managing NAT or bridging to virtual machine guests. Any external command that reloads iptables state ( such as running system-config-firewall) will overwrite the entries needed by libvirt. Consequently, after running any command or tool that changes the state of iptables, guests may lose access the network. To work around this issue, use the command 'service libvirtd reload' to restore libvirt's additional iptables rules.
Comment 30 Peter Krempa 2014-03-26 04:53:23 EDT
In RHEL 7 libvirt now uses firewalld which preserves firewall rules added by libvirt even through restarts:

[root@winston ~]# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    
[root@winston ~]# systemctl restart firewalld
[root@winston ~]# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    

I'm marking this as test-only and moving back to 7.0 as it's already fixed.
Comment 31 Jincheng Miao 2014-03-31 23:47:56 EDT
According to Comment 30 from Peter,
I tested it on RHEL-7.0-20140326.0, and iptables rules can get back after restarted firewalld. And the guest can ping to the target outside.

# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    

# systemctl restart firewalld 

# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24    
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535

The only different is there is redundant rule for tcp MASQUERADE of 192.168.122.0/24. But it not affect this virtual network.
Comment 32 zhoujunqin 2014-04-03 05:59:47 EDT
Seen  Comment 31,and i also get the same result,so set it as verified.
Comment 35 Ludek Smid 2014-06-13 07:34:11 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.