Bug 1460640 - virbr0 configured to be in "trusted" zone is not put in trusted zone at boot
virbr0 configured to be in "trusted" zone is not put in trusted zone at boot
Status: NEW
Product: Fedora
Classification: Fedora
Component: firewalld (Show other bugs)
26
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Eric Garver
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 1462223
  Show dependency treegraph
 
Reported: 2017-06-12 06:05 EDT by Timothy Redaelli
Modified: 2017-09-07 15:56 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1462223 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Network Manager log with level=TRACE (534.42 KB, text/x-vhdl)
2017-06-12 06:05 EDT, Timothy Redaelli
no flags Details
[logfile] for f26 and f25 (180.83 KB, application/x-gzip)
2017-06-16 06:37 EDT, Thomas Haller
no flags Details

  None (edit)
Description Timothy Redaelli 2017-06-12 06:05:54 EDT
Created attachment 1287007 [details]
Network Manager log with level=TRACE

Description of problem:


Version-Release number of selected component (if applicable):
NetworkManager 1.8.0-4.fc26

How reproducible:
100%

Steps to Reproduce:
1. Configure virbr0 (aka libvirt bridge) to be put in trusted zone:
 # firewall-cmd --permanent --zone=trusted --add-interface=virbr0
2. Reboot
3. Check if virbr0 is in trusted zone:
 # firewallctl info zone trusted

Actual results:
# firewallctl info zone trusted
trusted
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: 
  sources: 
  services: 
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 

Expected results:
#  firewallctl info zone trusted                                                                                   
trusted (active)
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: virbr0
  sources: 
  services: 
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

Additional info:

tredaell@graphite ~ % cat /etc/sysconfig/network-scripts/ifcfg-virbr0
DEVICE=virbr0
STP=yes
DELAY=2
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.122.1
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV4_DNS_PRIORITY=100
IPV6INIT=no
NAME=virbr0
UUID=793e6eb3-4108-4549-8149-73b069a55e5d
ONBOOT=no
ZONE=trusted
tredaell@graphite ~ %

tredaell@graphite ~ % nmcli connection show
NAME                                    UUID                                  TYPE             DEVICE     
enp0s31f6                               25b35999-0e7a-3204-bd33-9dcc7b85ce6a  802-3-ethernet   enp0s31f6 
virbr0                                  8b54b519-0a4e-42e4-9a04-c7dd1bfdc428  bridge           virbr0    
[...]     
virbr0                                  793e6eb3-4108-4549-8149-73b069a55e5d  bridge           --   
tredaell@graphite ~ %

As workaround you can do: "nmcli con up virbr0" and it will work.
Comment 1 Thomas Haller 2017-06-12 08:01:52 EDT
virbr0 is managed by libvirt. Usually, you don't want NetworkManager to interfere with it.

The virbr0 interface is not explicitly configured to be unmanaged for NetworkManager. Still,  NetworkManager notices that somebody else (libvirt) creates and configures the bridge. In this case, NM generates an in-memory connection that reflects the state on the device. It also means, NM will/should not configure the interface nor touch it.

You can modify the in-memory connection, which causes it to be persisted to disk (as you did).

When you issue `nmcli con up $NAME`, NM will activate the profile and fully manage the interface. But likely you don't want to do that, because libvirt is not expecting that.


Since firewallctl --persist basically configures the zone in the NetworkManager interface, ... this is not going to work for interfaces not actively managed by NM.

Thomas, what do you think how this should be solved?
Comment 2 Thomas Woerner 2017-06-12 08:29:29 EDT
Is is possible to identify this in-memory only state of a connection? Then firewalld might save the zone assignment itself. But this then also results in different zone settings in the NM and firewalld configuration and could also result in unexpected behaviour at some point.

What is the NM proposed way to handle these connections?

There should be only place to set and gather information about connections in my opinion. If this is not NetworkManager, what to use then?

I think libvirt should use NM to create and handle connections if NM is active. This is the way it is handling the firewall: If firewalld is active, then it uses firewalld, otherwise it is using ip*tables.
Comment 3 Thomas Haller 2017-06-12 09:06:01 EDT
(In reply to Thomas Woerner from comment #2)
> Is is possible to identify this in-memory only state of a connection? Then
> firewalld might save the zone assignment itself. 

No, it's currently not possible. We should add that.

> But this then also results
> in different zone settings in the NM and firewalld configuration and could
> also result in unexpected behaviour at some point.
> 
> What is the NM proposed way to handle these connections?

Leave them alone :)

I think it was a wrong decision to expose and generate these connections. It's only confusing. Probably we cannot change that now.

> There should be only place to set and gather information about connections
> in my opinion. If this is not NetworkManager, what to use then?

We maintain that NetworkManager is this place. If libvirt doesn't use it, there is no other place.


> I think libvirt should use NM to create and handle connections if NM is
> active.

I agree with that, we want NetworkManager to be the ubiquitous API to configure networking.

> This is the way it is handling the firewall: If firewalld is active,
> then it uses firewalld, otherwise it is using ip*tables.
Comment 4 Timothy Redaelli 2017-06-14 10:28:46 EDT
On Fedora 25 (NetworkManager 1.4.4-5.fc25) it has the "correct" behavior (for an user point of view), but on Fedora 26 (NetworkManager 1.8.0-5.fc26) it doesn't work anymore (without touching anything, just updated Fedora 25 to Fedora 26).

This could be seen as a "regression" since the behavior is changed.

I have a little Vagrantfile that show the different behaviors: 
https://gist.github.com/drizzt/cd3aae896854eec7986c947b6d09fa25

Just start the 2 vagrant profiles with `vagrant up`

Then you may show how the zone are shown after a reboot by using:

# vagrant reload f25 --provision-with show_zone
# vagrant reload f26 --provision-with show_zone

or by entering a machine with `vagrant ssh` and by running `sudo firewallctl info zone trusted` by hand
Comment 5 Thomas Haller 2017-06-16 06:37 EDT
Created attachment 1288311 [details]
[logfile] for f26 and f25

Logfiles using f25/f26 with the vagrant file from comment 4
Comment 6 Thomas Haller 2017-06-16 08:17:26 EDT
The issue is, that libvirt manages network devices on it's own. It is a very bad idea to have two managing deamon fighting over the same interface. That will end up in unpredictable behavior.

When NM sees that a device is managed by somebody else, it creates these external, in-memory connections, with the intend not to mess with the device (because clearly, NM isn't in charge).

When somebody modifies that connection, NM will persist the connection to disk and take over the interface [1]. It does so, because changing such a connection is taken as user decision to take over the interface.

Later, after restart, NM has indeed the previously persisted connection on disk. However, NM again sees that somebody else (libvirt) messes with the interface, and will not take over the interface. Again, NM creates an in-memory connection, despite having the previously stored connection on disk.


I don't think this is solvable, and I don't see how NM can do better here.
This is not restricted to firewall. The exact same thing happens, when the user tries to add an IP address:
  nmcli connection modify virbr0 +ipv4.addresses 192.168.7.4/24
at that moment [1], NM will indeed take over the interface. But after restart, it will not (and for good reasons).


@Thomas, what would you suggest to do about this? I think, either `firewallctl --persist` also grows support to talk to libvirt, or it just says: if the device is not managed by NM, it cannot persist the setting (like, NM also only supports firewalld, and no other implementations).

Firewalld could see that "unsaved" flag and fail the --persist command. "unsaved" is not exactly the same as those externally-managed interfaces, because you can create regular in-memory connections with
    nmcli connection add save no type ...
but an externally-managed connection is always unsaved, so it's probably good enough. A --persist on an in-memory connection doesn't make much sense anyway.


The same issues are in rhel-7.4


[1] actually, it will not, that is a bug that needs fixing. I will clone this bug to do that.
Comment 7 Eric Garver 2017-09-07 15:56:37 EDT
(In reply to Thomas Haller from comment #6)
[...]
> 
> @Thomas, what would you suggest to do about this? I think, either
> `firewallctl --persist` also grows support to talk to libvirt, or it just
> says: if the device is not managed by NM, it cannot persist the setting
> (like, NM also only supports firewalld, and no other implementations).
> 
> Firewalld could see that "unsaved" flag and fail the --persist command.
> "unsaved" is not exactly the same as those externally-managed interfaces,
> because you can create regular in-memory connections with
>     nmcli connection add save no type ...
> but an externally-managed connection is always unsaved, so it's probably
> good enough. A --persist on an in-memory connection doesn't make much sense
> anyway.

I think it's fair to reject the configuration if --persist is requested and NM tells firewalld that the interface is "unsaved". Of course this should give an intelligible error message of the "go fix it in NM first" variety. In the case that NM is not running firewalld will happily accept --persist.

I'll move this over to firewalld component then.

Note You need to log in before you can comment on or make changes to this bug.