Bug 1857820 - libvirt can't start virtual networks
Summary: libvirt can't start virtual networks
Keywords:
Status: CLOSED DUPLICATE of bug 1829090
Alias: None
Product: Fedora
Classification: Fedora
Component: firewalld
Version: 32
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Eric Garver
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-16 15:49 UTC by Bill Nottingham
Modified: 2020-07-16 17:53 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-16 17:53:12 UTC
Type: Bug


Attachments (Terms of Use)

Description Bill Nottingham 2020-07-16 15:49:07 UTC
Description of problem:

libvirt fails to start virtual networks

Error starting network 'default': error from service: changeZoneOfInterface: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed


Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/network.py", line 75, in start
    self._backend.create()
  File "/usr/lib64/python3.8/site-packages/libvirt.py", line 3174, in create
    if ret == -1: raise libvirtError ('virNetworkCreate() failed', net=self)
libvirt.libvirtError: error from service: changeZoneOfInterface: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed


(the 'line' number changes from time to time).


Version-Release number of selected component (if applicable):

libvirt-daemon-6.1.0-4.fc32.x86_64
firewalld-0.8.3-1.fc32.noarch
iptables-1.8.4-8.fc32.x86_64


How reproducible:
100%

Steps to Reproduce:
1. attempt to start VMs
2. fails due to no `default` virtual network
3. attempt to start default virtual network

Additional info:

Broke with recent updates, but downgrading does not fix it.

Comment 1 Bill Nottingham 2020-07-16 16:08:57 UTC
2020-07-16 12:02:18 DEBUG1: zone.getZones()
2020-07-16 12:02:18 DEBUG1: zone.changeZoneOfInterface('libvirt', 'virbr0')
2020-07-16 12:02:18 DEBUG1: Setting zone of interface 'virbr0' to 'libvirt'
2020-07-16 12:02:18 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 128, in execute
    self.fw.rules(backend_name, rules[backend_name])
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 874, in rules
    backend.set_rules(_rules, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 505, in set_rules
    raise ValueError("'%s %s' failed: %s" % (self._restore_command,
ValueError: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore: line 14 failed


2020-07-16 12:02:18 ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore: line 14 failed

2020-07-16 12:02:18 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 128, in execute
    self.fw.rules(backend_name, rules[backend_name])
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 874, in rules
    backend.set_rules(_rules, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 505, in set_rules
    raise ValueError("'%s %s' failed: %s" % (self._restore_command,
ValueError: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed


2020-07-16 12:02:18 ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed

2020-07-16 12:02:18 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/server/decorators.py", line 68, in dbus_handle_exceptions
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/firewall/server/firewalld.py", line 1185, in changeZoneOfInterface
    _zone = self.fw.zone.change_zone_of_interface(zone, interface, sender)
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_zone.py", line 450, in change_zone_of_interface
    _zone = self.add_interface(zone, interface, sender)
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_zone.py", line 428, in add_interface
    transaction.execute(True)
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 173, in execute
    raise FirewallError(errors.COMMAND_FAILED, errorMsg)
firewall.errors.FirewallError: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed


2020-07-16 12:02:18 ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore: line 14 failed

Comment 2 Daniel Berrangé 2020-07-16 16:10:05 UTC
This error is coming from firewalld, when libvirt invokes the changeZoneOfInterface DBus call.

We've recently seen issues with failures due to historical workarounds for docker, so I'm guessing this is probably be a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1829090

Comment 3 Eric Garver 2020-07-16 16:33:45 UTC
Can you check the firewalld log for errors? It may help to enable debug via /etc/sysconfig/firewalld.

Also try enabling IndividualCalls=yes in /etc/firewalld/firewalld.conf to get a better pointer to the failed iptables command.

Comment 4 Bill Nottingham 2020-07-16 17:43:29 UTC
Example errors from a simple service restart:
2020-07-16 13:38:33 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.


2020-07-16 13:38:33 ERROR: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.

2020-07-16 13:38:34 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 128, in execute
    self.fw.rules(backend_name, rules[backend_name])
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 872, in rules
    raise msg
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.


2020-07-16 13:38:34 ERROR: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.


When libvirt tries to start the default network:

2020-07-16 13:38:33 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.


2020-07-16 13:38:33 ERROR: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.

2020-07-16 13:38:34 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 128, in execute
    self.fw.rules(backend_name, rules[backend_name])
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 872, in rules
    raise msg
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.


2020-07-16 13:38:34 ERROR: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i virbr0 -g PRE_libvirt' failed: iptables: No chain/target/match by that name.

Comment 5 Bill Nottingham 2020-07-16 17:44:39 UTC
Take 2: service restart even when libvirtd isn't running yields:

2020-07-16 13:41:58 DEBUG1: Setting zone of interface 'wlp4s0' to 'FedoraWorkstation'
2020-07-16 13:41:58 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i wlp4s0 -g PRE_FedoraWorkstation' failed: iptables: No chain/target/match by that name.


2020-07-16 13:41:58 ERROR: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i wlp4s0 -g PRE_FedoraWorkstation' failed: iptables: No chain/target/match by that name.

2020-07-16 13:41:58 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/firewall/core/fw_transaction.py", line 128, in execute
    self.fw.rules(backend_name, rules[backend_name])
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 872, in rules
    raise msg
  File "/usr/lib/python3.8/site-packages/firewall/core/fw.py", line 862, in rules
    backend.set_rule(rule, self._log_denied)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 536, in set_rule
    output = self.__run(rule)
  File "/usr/lib/python3.8/site-packages/firewall/core/ipXtables.py", line 197, in __run
    raise ValueError("'%s %s' failed: %s" % (self._command,
ValueError: '/usr/sbin/iptables -w10 -I PREROUTING_ZONES 1 -t raw -i wlp4s0 -g PRE_FedoraWorkstation' failed: iptables: No chain/target/match by that name.

Comment 6 Bill Nottingham 2020-07-16 17:53:12 UTC
Yep, it's https://bugzilla.redhat.com/show_bug.cgi?id=1829090.

*** This bug has been marked as a duplicate of bug 1829090 ***

Comment 7 Bill Nottingham 2020-07-16 17:53:53 UTC
Since it's technically a firewalld config, might be worth changing the component on https://bugzilla.redhat.com/show_bug.cgi?id=1829090 so it comes up easier in a search.


Note You need to log in before you can comment on or make changes to this bug.