Bug 1603115 - Backport clean-traffic-gateway into RHEL 7.6
Summary: Backport clean-traffic-gateway into RHEL 7.6
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Martin Kletzander
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 1009608
TreeView+ depends on / blocked
 
Reported: 2018-07-19 08:22 UTC by Ales Musil
Modified: 2018-10-30 09:59 UTC (History)
13 users (show)

Fixed In Version: libvirt-4.5.0-5.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-30 09:58:24 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:3113 None None None 2018-10-30 09:59:15 UTC

Description Ales Musil 2018-07-19 08:22:46 UTC
Description of problem:

Within oVirt we would like to use the new clean-traffic-gateway network filter [1] as soon as possible. We would like the filter to be backported into RHEL 7.6. 

[1] https://libvirt.org/git/?p=libvirt.git;a=commit;h=ac01fbc90b7eb4ccc7a6140d618d1a3859365155

Comment 2 Rob Young 2018-07-19 13:33:51 UTC
@mtessun can you help move this along with the appropriate RHEL subsystems lead?

Comment 3 Jaroslav Suchanek 2018-07-19 20:18:59 UTC
This upstream commit is sufficient?

commit ac01fbc90b7eb4ccc7a6140d618d1a3859365155
Author:     Ales Musil <amusil@redhat.com>
AuthorDate: Wed Jul 18 10:33:03 2018 +0200
Commit:     Martin Kletzander <mkletzan@redhat.com>
CommitDate: Wed Jul 18 14:03:47 2018 +0200

    examples: Add clean-traffic-gateway into nwfilters
    
    The filter purpose is to simulate isolated private VLAN.
    
    The behavior can be achieved by limiting network traffic
    to traffic between VM and gateway. Because there is no
    concept of the PVLAN in the linux bridge.
    
    The filter also contains parts from clean-traffic
    to prevent VM from spoofing its IP and MAC address.
    
    To use this filter the user just needs to set
    the GATEWAY_MAC variable to gateway MAC address.
    
    Signed-off-by: Ales Musil <amusil@redhat.com>
    Reviewed-by: Martin Kletzander <mkletzan@redhat.com>

Comment 4 Dan Kenigsberg 2018-07-22 08:19:24 UTC
yes, this is exactly what we'd like to see in downstream.

Comment 5 Michael Burman 2018-07-22 08:54:40 UTC
RHEV QE will be glad to test this in the RHEV context, would you consider to clone it to downstream?

Comment 6 Martin Tessun 2018-07-23 16:27:06 UTC
As it is past devel freeze for RHEL 7.6 adding exception flag here as well.

Comment 11 yalzhang@redhat.com 2018-08-01 09:22:39 UTC
Test on libvirt-4.5.0-5.el7.x86_64, and I found the interface refer to this filter can not get ip address by dhcp, if it is expected? Please help to confirm, Thank you!

1) start a vm with default network, the gateway is virbr0 with mac '52:54:00:15:a5:40', and the virbr0 also act as dhcp server running dnsmasq

# virsh dumpxml rhel | grep /interface -B10
    <interface type='network'>
      <mac address='52:54:00:62:f7:c3'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet1'/>
      <model type='rtl8139'/>
      <filterref filter='clean-traffic-gateway'>
        <parameter name='GATEWAY_MAC' value='52:54:00:15:a5:40'/>
      </filterref>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

2) log in guest, and check it can not get ip address:
# dhclient -d ens3
Internet Systems Consortium DHCP Client 4.2.5
Copyright 2004-2013 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/ens3/52:54:00:62:f7:c3
Sending on   LPF/ens3/52:54:00:62:f7:c3
Sending on   Socket/fallback
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 3 (xid=0x6ed31669)
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 6 (xid=0x6ed31669)
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 12 (xid=0x6ed31669)
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 13 (xid=0x6ed31669)
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 11 (xid=0x6ed31669)
DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 16 (xid=0x6ed31669)
No DHCPOFFERS received.
No working leases in persistent database - sleeping.

3) but check the ebtables rules on host, the ip is learnt as '192.168.122.208'
# ebtables -t nat -L
...
Bridge chain: libvirt-I-vnet1, entries: 9, policy: ACCEPT
-j I-vnet1-mac
-p IPv4 -j I-vnet1-ipv4-ip
-p ARP -j I-vnet1-arp-mac
-p ARP -j I-vnet1-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet1-rarp
-d 52:54:0:15:a5:40 -j ACCEPT 
-p 0x835 -j ACCEPT 
-j DROP 
...
Bridge chain: libvirt-O-vnet1, entries: 4, policy: ACCEPT
-p ARP -j ACCEPT 
-p 0x8035 -j O-vnet1-rarp
-s 52:54:0:15:a5:40 -j ACCEPT 
-j DROP 

Bridge chain: I-vnet1-mac, entries: 2, policy: ACCEPT
-s 52:54:0:62:f7:c3 -j RETURN 
-j DROP 

Bridge chain: I-vnet1-ipv4-ip, entries: 3, policy: ACCEPT
-p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
-p IPv4 --ip-src 192.168.122.208 -j RETURN 
-j DROP 

Bridge chain: I-vnet1-arp-mac, entries: 2, policy: ACCEPT
-p ARP --arp-mac-src 52:54:0:62:f7:c3 -j RETURN 
-j DROP 

Bridge chain: I-vnet1-arp-ip, entries: 2, policy: ACCEPT
-p ARP --arp-ip-src 192.168.122.208 -j RETURN 
-j DROP 

Bridge chain: I-vnet1-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -s 52:54:0:62:f7:c3 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:62:f7:c3 --arp-mac-dst 52:54:0:62:f7:c3 -j ACCEPT 
-j DROP 

Bridge chain: O-vnet1-rarp, entries: 2, policy: ACCEPT
-p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:62:f7:c3 --arp-mac-dst 52:54:0:62:f7:c3 -j ACCEPT 
-j DROP 


Also test with linux bridge, the result is the same. The rules could learn the dhcp ip address, but guest can not get that ip. When I change to "clean-traffic", the guest can get ip address successfully.


Other test about virsh cmd, all result is as expected.
1) list
# virsh nwfilter-list
 UUID                                  Name                 
------------------------------------------------------------------
...
 22746188-0acf-4389-910e-7588f12b5c0a  clean-traffic-gateway

2) dump 
# virsh nwfilter-dumpxml clean-traffic-gateway
<filter name='clean-traffic-gateway' chain='root'>
  <uuid>22746188-0acf-4389-910e-7588f12b5c0a</uuid>
  <filterref filter='no-mac-spoofing'/>
  <filterref filter='no-ip-spoofing'/>
  <filterref filter='no-arp-spoofing'/>
  <rule action='accept' direction='inout' priority='-500'>
    <mac protocolid='arp'/>
  </rule>
  <rule action='accept' direction='in' priority='500'>
    <mac srcmacaddr='$GATEWAY_MAC'/>
  </rule>
  <rule action='accept' direction='out' priority='500'>
    <mac dstmacaddr='$GATEWAY_MAC'/>
  </rule>
  <filterref filter='no-other-l2-traffic'/>
  <filterref filter='qemu-announce-self'/>
</filter>

3) validate xml
# virsh nwfilter-dumpxml clean-traffic-gateway > clean-traffic-gateway.xml
# virt-xml-validate clean-traffic-gateway.xml
clean-traffic-gateway.xml validates

Comment 12 Ales Musil 2018-08-01 11:23:27 UTC
(In reply to yalzhang@redhat.com from comment #11)
> Test on libvirt-4.5.0-5.el7.x86_64, and I found the interface refer to this
> filter can not get ip address by dhcp, if it is expected? Please help to
> confirm, Thank you!
> 
> 1) start a vm with default network, the gateway is virbr0 with mac
> '52:54:00:15:a5:40', and the virbr0 also act as dhcp server running dnsmasq
> 
> # virsh dumpxml rhel | grep /interface -B10
>     <interface type='network'>
>       <mac address='52:54:00:62:f7:c3'/>
>       <source network='default' bridge='virbr0'/>
>       <target dev='vnet1'/>
>       <model type='rtl8139'/>
>       <filterref filter='clean-traffic-gateway'>
>         <parameter name='GATEWAY_MAC' value='52:54:00:15:a5:40'/>
>       </filterref>
>       <alias name='net0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
> 
> 2) log in guest, and check it can not get ip address:
> # dhclient -d ens3
> Internet Systems Consortium DHCP Client 4.2.5
> Copyright 2004-2013 Internet Systems Consortium.
> All rights reserved.
> For info, please visit https://www.isc.org/software/dhcp/
> 
> Listening on LPF/ens3/52:54:00:62:f7:c3
> Sending on   LPF/ens3/52:54:00:62:f7:c3
> Sending on   Socket/fallback
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 3 (xid=0x6ed31669)
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 6 (xid=0x6ed31669)
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 12 (xid=0x6ed31669)
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 13 (xid=0x6ed31669)
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 11 (xid=0x6ed31669)
> DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 16 (xid=0x6ed31669)
> No DHCPOFFERS received.
> No working leases in persistent database - sleeping.
> 
> 3) but check the ebtables rules on host, the ip is learnt as
> '192.168.122.208'
> # ebtables -t nat -L
> ...
> Bridge chain: libvirt-I-vnet1, entries: 9, policy: ACCEPT
> -j I-vnet1-mac
> -p IPv4 -j I-vnet1-ipv4-ip
> -p ARP -j I-vnet1-arp-mac
> -p ARP -j I-vnet1-arp-ip
> -p ARP -j ACCEPT 
> -p 0x8035 -j I-vnet1-rarp
> -d 52:54:0:15:a5:40 -j ACCEPT 
> -p 0x835 -j ACCEPT 
> -j DROP 
> ...
> Bridge chain: libvirt-O-vnet1, entries: 4, policy: ACCEPT
> -p ARP -j ACCEPT 
> -p 0x8035 -j O-vnet1-rarp
> -s 52:54:0:15:a5:40 -j ACCEPT 
> -j DROP 
> 
> Bridge chain: I-vnet1-mac, entries: 2, policy: ACCEPT
> -s 52:54:0:62:f7:c3 -j RETURN 
> -j DROP 
> 
> Bridge chain: I-vnet1-ipv4-ip, entries: 3, policy: ACCEPT
> -p IPv4 --ip-src 0.0.0.0 --ip-proto udp -j RETURN 
> -p IPv4 --ip-src 192.168.122.208 -j RETURN 
> -j DROP 
> 
> Bridge chain: I-vnet1-arp-mac, entries: 2, policy: ACCEPT
> -p ARP --arp-mac-src 52:54:0:62:f7:c3 -j RETURN 
> -j DROP 
> 
> Bridge chain: I-vnet1-arp-ip, entries: 2, policy: ACCEPT
> -p ARP --arp-ip-src 192.168.122.208 -j RETURN 
> -j DROP 
> 
> Bridge chain: I-vnet1-rarp, entries: 2, policy: ACCEPT
> -p 0x8035 -s 52:54:0:62:f7:c3 -d Broadcast --arp-op Request_Reverse
> --arp-ip-src 0.0.0.0 --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:62:f7:c3
> --arp-mac-dst 52:54:0:62:f7:c3 -j ACCEPT 
> -j DROP 
> 
> Bridge chain: O-vnet1-rarp, entries: 2, policy: ACCEPT
> -p 0x8035 -d Broadcast --arp-op Request_Reverse --arp-ip-src 0.0.0.0
> --arp-ip-dst 0.0.0.0 --arp-mac-src 52:54:0:62:f7:c3 --arp-mac-dst
> 52:54:0:62:f7:c3 -j ACCEPT 
> -j DROP 
> 
> 
> Also test with linux bridge, the result is the same. The rules could learn
> the dhcp ip address, but guest can not get that ip. When I change to
> "clean-traffic", the guest can get ip address successfully.
> 
> 
> Other test about virsh cmd, all result is as expected.
> 1) list
> # virsh nwfilter-list
>  UUID                                  Name                 
> ------------------------------------------------------------------
> ...
>  22746188-0acf-4389-910e-7588f12b5c0a  clean-traffic-gateway
> 
> 2) dump 
> # virsh nwfilter-dumpxml clean-traffic-gateway
> <filter name='clean-traffic-gateway' chain='root'>
>   <uuid>22746188-0acf-4389-910e-7588f12b5c0a</uuid>
>   <filterref filter='no-mac-spoofing'/>
>   <filterref filter='no-ip-spoofing'/>
>   <filterref filter='no-arp-spoofing'/>
>   <rule action='accept' direction='inout' priority='-500'>
>     <mac protocolid='arp'/>
>   </rule>
>   <rule action='accept' direction='in' priority='500'>
>     <mac srcmacaddr='$GATEWAY_MAC'/>
>   </rule>
>   <rule action='accept' direction='out' priority='500'>
>     <mac dstmacaddr='$GATEWAY_MAC'/>
>   </rule>
>   <filterref filter='no-other-l2-traffic'/>
>   <filterref filter='qemu-announce-self'/>
> </filter>
> 
> 3) validate xml
> # virsh nwfilter-dumpxml clean-traffic-gateway > clean-traffic-gateway.xml
> # virt-xml-validate clean-traffic-gateway.xml
> clean-traffic-gateway.xml validates

Yes, this is expected. As the DHCPDISCOVER is sent to ff:ff:ff:ff:ff:ff which is broadcast MAC address. To get it working you would have to add <parameter name='GATEWAY_MAC' value='ff:ff:ff:ff:ff:ff'/> into the filterref as well.

Comment 13 yalzhang@redhat.com 2018-08-01 13:09:32 UTC
Thank you for the timely reply. I have tested it, and it can get ip and works well. 
Do you think it is necessary to solidify the "<parameter name='GATEWAY_MAC' value='ff:ff:ff:ff:ff:ff'/>" into the clean-traffic-gateway definition? So we don't need to specify it in the guest xml(I don't know if there is such scenario that should block broadcast traffic). Or we can just add it into document(I'm not sure it is necessary)
And I think it is better to have document for this new introduced nwfilter in /usr/share/doc/libvirt-docs-4.5.0/html/formatnwfilter.html.

Thank you~

1. default network
1) Start 2 vm connected to virbr0 with nwfilter as below:
# virsh dumpxml rh | grep /interface -B11
    <interface type='network'>
      <mac address='52:54:00:af:18:5c'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet2'/>
      <model type='rtl8139'/>
      <filterref filter='clean-traffic-gateway'>
        <parameter name='GATEWAY_MAC' value='ff:ff:ff:ff:ff:ff'/>
        <parameter name='GATEWAY_MAC' value='52:54:00:15:a5:40'/>
      </filterref>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

2) check the ebtables rules
Bridge chain: libvirt-I-vnet2, entries: 10, policy: ACCEPT
-j I-vnet2-mac
-p IPv4 -j I-vnet2-ipv4-ip
-p ARP -j I-vnet2-arp-mac
-p ARP -j I-vnet2-arp-ip
-p ARP -j ACCEPT 
-p 0x8035 -j I-vnet2-rarp
-d Broadcast -j ACCEPT   ======> this is for the  'ff:ff:ff:ff:ff:ff'
-d 52:54:0:15:a5:40 -j ACCEPT  ===> this is for the '52:54:00:15:a5:40'
-p 0x835 -j ACCEPT 
-j DROP 

3) check the connectivity
the vm connected to the same bridge can access to the outside, but can not connect to each other

2. test host bridge, the result is the same.

Comment 14 Martin Kletzander 2018-08-03 13:44:37 UTC
As the nwfilter can be used only by someone who actually reads what it does and understands it I don't see the point in describing it in detail anywhere except the definition itself.  Also the filter might be used for networks without DHCP in this case.  I think this is fine.  Maybe worth asking the original reequester if that's enough for them as well.  Ales?

Comment 15 Ales Musil 2018-08-09 05:58:40 UTC
Seems fine to me.

Comment 17 errata-xmlrpc 2018-10-30 09:58:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113


Note You need to log in before you can comment on or make changes to this bug.