Bug 1829090 - Error starting domain: Requested operation is not valid: network 'default' is not active
Summary: Error starting domain: Requested operation is not valid: network 'default' is...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: firewalld
Version: 32
Hardware: x86_64
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Eric Garver
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1857820 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-28 21:00 UTC by Paul Scott
Modified: 2020-08-09 07:47 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-18 09:17:55 UTC
Type: Bug


Attachments (Terms of Use)

Description Paul Scott 2020-04-28 21:00:03 UTC
Description of problem:

After upgrade to Fedora 32 virtual manager will not start existing virtual machine.

Error starting domain: Requested operation is not valid: network 'default' is not active

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)

In machine settings, Network Source "Virtual network 'default': NAT (inactive) 

Version-Release number of selected component (if applicable):
2.2.1

How reproducible:
Every time

Steps to Reproduce:
1. Open virt-manager, enter password to elevate privileges
2. Select VM to run
3. Start VM

Actual results:
Error above

Expected results:
Machine should start

Additional info:

Comment 1 Kyle Cianflone 2020-04-28 22:22:20 UTC
I am seeing the same issue, Testing seems to show some kind of an issue with NAT networks and nftables. Problems persist on attempting to create a new NAT network.


Error starting network 'default': error from service: changeZoneOfInterface: COMMAND_FAILED: 'python-nftables' failed: 
JSON blob:
{"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "udp", "field": "dport"}}, "op": "==", "right": 67}}, {"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["new", "untracked"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "udp", "field": "dport"}}, "op": "==", "right": 547}}, {"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["new", "untracked"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "tcp", "field": "dport"}}, "op": "==", "right": 53}}, {"match": {"left": {"ct": {"key": "state"}}

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/network.py", line 75, in start
    self._backend.create()
  File "/usr/lib64/python3.8/site-packages/libvirt.py", line 3174, in create
    if ret == -1: raise libvirtError ('virNetworkCreate() failed', net=self)
libvirt.libvirtError: error from service: changeZoneOfInterface: COMMAND_FAILED: 'python-nftables' failed: 
JSON blob:
{"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "udp", "field": "dport"}}, "op": "==", "right": 67}}, {"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["new", "untracked"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "udp", "field": "dport"}}, "op": "==", "right": 547}}, {"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": {"set": ["new", "untracked"]}}}, {"accept": null}]}}}, {"add": {"rule": {"family": "inet", "table": "firewalld", "chain": "filter_IN_libvirt_allow", "expr": [{"match": {"left": {"payload": {"protocol": "tcp", "field": "dport"}}, "op": "==", "right": 53}}, {"match": {"left": {"ct": {"key": "state"}}

Comment 2 Tadas Giniotis 2020-05-26 07:39:39 UTC
I get the same error when running a VM from virt-manager, and when trying to bring up the default network (virbr0), I get the following:

$: sudo virsh net-start default

error: Failed to start network default
error: Error desde el servicio: changeZoneOfInterface: COMMAND_FAILED: '{"%%ZONE_INTERFACE%%": null, "chain": "raw_PREROUTING_ZONES", "expr": [{"match": {"left": {"meta": {"key": "iifname"}}, "op": "==", "right": "virbr0"}}, {"goto": {"target": "raw_PRE_trusted"}}], "family": "inet", "table": "firewalld"}'

Comment 3 Jason Montleon 2020-05-26 16:32:19 UTC
I started seeing this today.

After a lot of digging I noticed on boot an error in the journal:
"May 26 11:19:08 jmontleo.usersys.redhat.com firewalld[2850]: ERROR: ZONE_CONFLICT: 'docker0' already bound to a zone"

This ended up being because I used the workaround of adding docker0 to the trusted zone for:
https://bugzilla.redhat.com/show_bug.cgi?id=1817022

It looks like a new moby-engine package went to stable a few days ago which added a docker.xml zone file in /usr/lib/firewalld/zones.

I ended up having to delete /etc/firewalld/zones/trusted.xml to remove the workaround. One this was done and I rebooted libvirt (and my ethernet interface for that matter!) ended up in the correct zones.

It is not great that firewalld stops adding interfaces to zones and becomes pretty unusable when it hits a zone conflict like that. Might be better to move this to firewalld to see if they can do anything about that.

Comment 4 Radosław Ulatowski 2020-05-26 21:03:20 UTC
I have same problem as described.

I've tried to disect this issue and found out that removing docker (moby-engine / containerd) fixed this. One thing i noticed is that when docker is installed new br-... interface is created (ifconfig) and libvirt is unable to start any network.

This issue (with different errors messages) occurs on both iptables and nftables backends for firewalld. I've changed to iptables to fix containers access to external network and I've reversed to nftables for testing.

Comment 5 Tadas Giniotis 2020-05-27 08:03:15 UTC
As suggested in comment #3 - deleting the "/etc/firewalld/zones/trusted.xml" file and restarting the OS fixed the issue of VMs not launching in virt-manager.

Comment 6 Michael DePaulo 2020-06-17 18:06:20 UTC
+1 to deleting /etc/firewalld/zones/trusted.xml for aforementioned reasons.

I did not need to reboot, just restart firewalld service, then restart libvirtd service.

Comment 7 Daniel Berrangé 2020-06-18 09:17:55 UTC
Closing since evidence points towards the problem being firewalld error due to bad docker zone config.

Comment 8 Craig Ringer 2020-07-01 04:45:13 UTC
It's not completely clear why the added trusted.xml is the cause here. Dut docker/moby is certainly doing something wrong. My libvirt networking broke when I updated fedora 32 - it was fine when I first installed, so breakage must be after the fedora 32 release.

Interestingly I see

# firewall-cmd --list-services
You're performing an operation over default zone ('FedoraWorkstation'),
but your connections/interfaces are in zone 'docker' (see --get-active-zones)
You most likely need to use --zone=docker option.

dhcpv6-client mdns postgresql samba-client ssh

# firewall-cmd --get-active-zones
docker
  interfaces: docker0

# firewall-cmd --get-zones
FedoraServer FedoraWorkstation block dmz docker drop external home internal libvirt nm-shared public trusted work

# firewall-cmd --get-zone-of-interface docker0
docker

# firewall-cmd --get-zone-of-interface lo
no zone

# firewall-cmd --list-interfaces --zone docker
docker0


So it seems like by putting the bridge in the trusted 'docker' zone, moby has inadvertently hijacked zoning for the system.

Comment 9 Bill Nottingham 2020-07-16 17:53:12 UTC
*** Bug 1857820 has been marked as a duplicate of this bug. ***

Comment 10 Eric Garver 2020-07-16 19:53:27 UTC
(In reply to Craig Ringer from comment #8)
> So it seems like by putting the bridge in the trusted 'docker' zone, moby
> has inadvertently hijacked zoning for the system.

See this upstream moby change: https://github.com/moby/libnetwork/pull/2548


Note You need to log in before you can comment on or make changes to this bug.