Bug 1138807 - Unable to restart vdsmd service due to stale libvirt nwfilter
Summary: Unable to restart vdsmd service due to stale libvirt nwfilter
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: vdsm
Version: 3.5
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.5.0
Assignee: Petr Horáček
QA Contact: Meni Yakove
URL:
Whiteboard: network
: 1141568 (view as bug list)
Depends On:
Blocks: 1154665
TreeView+ depends on / blocked
 
Reported: 2014-09-05 16:32 UTC by Adam Litke
Modified: 2016-02-10 19:38 UTC (History)
13 users (show)

Fixed In Version: v4.16.4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-10-17 12:29:22 UTC
oVirt Team: Network
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 32691 0 master MERGED revert: libvirt is happy with redefinition of an existing nwfilter Never
oVirt gerrit 32696 0 ovirt-3.5 MERGED revert: libvirt is happy with redefinition of an existing nwfilter Never
oVirt gerrit 32697 0 ovirt-3.4 MERGED revert: libvirt is happy with redefinition of an existing nwfilter Never

Description Adam Litke 2014-09-05 16:32:38 UTC
Description of problem:

The vdsmd service cannot be restarted due to a stale libvirt nwfilter.


Version-Release number of selected component (if applicable):

vdsm-4.16.2-9.git5a23b22.fc20.x86_64 from ovirt-3.5-snapshot
libvirt-daemon-1.2.7-2.fc20.x86_64 from fedora virt-preview repo
 
How reproducible: Always

Steps to Reproduce:
1. Attempt to restart vdsmd

Actual results:
$ sudo service vdsmd restart
Redirecting to /bin/systemctl restart  vdsmd.service
Job for vdsmd.service failed. See 'systemctl status vdsmd.service' and 'journalctl -xn' for details.
[alitke@lager ~]$ systemctl status vdsmd.service
vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled)
   Active: failed (Result: start-limit) since Fri 2014-09-05 12:23:46 EDT; 29s ago
  Process: 20725 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS)
  Process: 19767 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=0/SUCCESS)
  Process: 20918 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=1/FAILURE)
 Main PID: 19767 (code=exited, status=0/SUCCESS)

Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail....
Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service holdoff time over....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Stopping Virtual Desktop Server....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Starting Virtual Desktop Server....
Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service start request rep....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail....
Hint: Some lines were ellipsized, use -l to show in full.

$ sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
[sudo] password for alitke: 
vdsm: Running mkdirs
vdsm: Running configure_coredump
vdsm: Running configure_vdsm_logs
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running check_is_configured
libvirt is already configured for vdsm
vdsm: Running validate_configuration
SUCCESS: ssl configured to true. No conflicts
vdsm: Running prepare_transient_repository
vdsm: Running syslog_available
vdsm: Running nwfilter
libvirt: Network Filter Driver error : operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a
Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 209, in main
    return tool_command[cmd]["command"](*args)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 38, in main
    NoMacSpoofingFilter().defineNwFilter(conn)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 60, in defineNwFilter
    nwFilter = conn.nwfilterDefineXML(self.buildFilterXml())
  File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 111, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4004, in nwfilterDefineXML
    if ret is None:raise libvirtError('virNWFilterDefineXML() failed', conn=self)
libvirtError: operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a
vdsm: stopped during execute nwfilter task (task returned with error code 1).


Expected results:
vdsmd restart is successful

Additional info:
To work around this problem I need to do the following command sequence:
sudo virsh nwfilter-undefine vdsm-no-mac-spoofing
sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
sudo virsh nwfilter-undefine vdsm-no-mac-spoofing
sudo service vdsmd restart

Comment 1 Dan Kenigsberg 2014-09-06 18:17:42 UTC
Ouch; Seems to be an intended libvirt change of behavior due to bug 1077009.

It's possible to reverted the following patch, but it would re-introduce noisy 

commit 657d413d79ef736c425c9bc7112d8a710083ea9b
Author: Dan Kenigsberg <danken>
Date:   Thu Oct 31 21:34:53 2013 +0000

    libvirt is happy with redefinition of an existing nwfilter
    
    Change-Id: Id253c66df8296613b6a2bc3fc058ef2754702964
    Reviewed-on: http://gerrit.ovirt.org/20788

Comment 2 Dan Kenigsberg 2014-09-14 21:31:34 UTC
*** Bug 1141568 has been marked as a duplicate of this bug. ***

Comment 3 Meni Yakove 2014-09-15 08:42:09 UTC
vdsm-4.16.3-3.el6ev.beta.x86_64

Comment 4 Marian Krcmarik 2014-09-17 12:37:55 UTC
(In reply to Meni Yakove from comment #3)
> vdsm-4.16.3-3.el6ev.beta.x86_64

Hi,
The "Fixed in Version" field says "v4.16.4" but you verified on 4.16.3-3? is that a downstream build or upstream? I can still observe the problem on 4.16.3-3 downstream build.

Comment 5 Sandro Bonazzola 2014-10-17 12:29:22 UTC
oVirt 3.5 has been released and should include the fix for this issue.


Note You need to log in before you can comment on or make changes to this bug.