Description of problem:
The vdsmd service cannot be restarted due to a stale libvirt nwfilter.
Version-Release number of selected component (if applicable):
vdsm-4.16.2-9.git5a23b22.fc20.x86_64 from ovirt-3.5-snapshot
libvirt-daemon-1.2.7-2.fc20.x86_64 from fedora virt-preview repo
How reproducible: Always
Steps to Reproduce:
1. Attempt to restart vdsmd
Actual results:
$ sudo service vdsmd restart
Redirecting to /bin/systemctl restart vdsmd.service
Job for vdsmd.service failed. See 'systemctl status vdsmd.service' and 'journalctl -xn' for details.
[alitke@lager ~]$ systemctl status vdsmd.service
vdsmd.service - Virtual Desktop Server Manager
Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled)
Active: failed (Result: start-limit) since Fri 2014-09-05 12:23:46 EDT; 29s ago
Process: 20725 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS)
Process: 19767 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=0/SUCCESS)
Process: 20918 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=1/FAILURE)
Main PID: 19767 (code=exited, status=0/SUCCESS)
Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail....
Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service holdoff time over....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Stopping Virtual Desktop Server....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Starting Virtual Desktop Server....
Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service start request rep....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop....
Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail....
Hint: Some lines were ellipsized, use -l to show in full.
$ sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
[sudo] password for alitke:
vdsm: Running mkdirs
vdsm: Running configure_coredump
vdsm: Running configure_vdsm_logs
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running check_is_configured
libvirt is already configured for vdsm
vdsm: Running validate_configuration
SUCCESS: ssl configured to true. No conflicts
vdsm: Running prepare_transient_repository
vdsm: Running syslog_available
vdsm: Running nwfilter
libvirt: Network Filter Driver error : operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a
Traceback (most recent call last):
File "/usr/bin/vdsm-tool", line 209, in main
return tool_command[cmd]["command"](*args)
File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 38, in main
NoMacSpoofingFilter().defineNwFilter(conn)
File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 60, in defineNwFilter
nwFilter = conn.nwfilterDefineXML(self.buildFilterXml())
File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 111, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4004, in nwfilterDefineXML
if ret is None:raise libvirtError('virNWFilterDefineXML() failed', conn=self)
libvirtError: operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a
vdsm: stopped during execute nwfilter task (task returned with error code 1).
Expected results:
vdsmd restart is successful
Additional info:
To work around this problem I need to do the following command sequence:
sudo virsh nwfilter-undefine vdsm-no-mac-spoofing
sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
sudo virsh nwfilter-undefine vdsm-no-mac-spoofing
sudo service vdsmd restart
Ouch; Seems to be an intended libvirt change of behavior due to bug 1077009.
It's possible to reverted the following patch, but it would re-introduce noisy
commit 657d413d79ef736c425c9bc7112d8a710083ea9b
Author: Dan Kenigsberg <danken>
Date: Thu Oct 31 21:34:53 2013 +0000
libvirt is happy with redefinition of an existing nwfilter
Change-Id: Id253c66df8296613b6a2bc3fc058ef2754702964
Reviewed-on: http://gerrit.ovirt.org/20788
(In reply to Meni Yakove from comment #3)
> vdsm-4.16.3-3.el6ev.beta.x86_64
Hi,
The "Fixed in Version" field says "v4.16.4" but you verified on 4.16.3-3? is that a downstream build or upstream? I can still observe the problem on 4.16.3-3 downstream build.
Description of problem: The vdsmd service cannot be restarted due to a stale libvirt nwfilter. Version-Release number of selected component (if applicable): vdsm-4.16.2-9.git5a23b22.fc20.x86_64 from ovirt-3.5-snapshot libvirt-daemon-1.2.7-2.fc20.x86_64 from fedora virt-preview repo How reproducible: Always Steps to Reproduce: 1. Attempt to restart vdsmd Actual results: $ sudo service vdsmd restart Redirecting to /bin/systemctl restart vdsmd.service Job for vdsmd.service failed. See 'systemctl status vdsmd.service' and 'journalctl -xn' for details. [alitke@lager ~]$ systemctl status vdsmd.service vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled) Active: failed (Result: start-limit) since Fri 2014-09-05 12:23:46 EDT; 29s ago Process: 20725 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS) Process: 19767 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=0/SUCCESS) Process: 20918 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=1/FAILURE) Main PID: 19767 (code=exited, status=0/SUCCESS) Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop.... Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail.... Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service holdoff time over.... Sep 05 12:23:46 lager.alitke.net systemd[1]: Stopping Virtual Desktop Server.... Sep 05 12:23:46 lager.alitke.net systemd[1]: Starting Virtual Desktop Server.... Sep 05 12:23:46 lager.alitke.net systemd[1]: vdsmd.service start request rep.... Sep 05 12:23:46 lager.alitke.net systemd[1]: Failed to start Virtual Desktop.... Sep 05 12:23:46 lager.alitke.net systemd[1]: Unit vdsmd.service entered fail.... Hint: Some lines were ellipsized, use -l to show in full. $ sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start [sudo] password for alitke: vdsm: Running mkdirs vdsm: Running configure_coredump vdsm: Running configure_vdsm_logs vdsm: Running run_init_hooks vdsm: Running gencerts vdsm: Running check_is_configured libvirt is already configured for vdsm vdsm: Running validate_configuration SUCCESS: ssl configured to true. No conflicts vdsm: Running prepare_transient_repository vdsm: Running syslog_available vdsm: Running nwfilter libvirt: Network Filter Driver error : operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 209, in main return tool_command[cmd]["command"](*args) File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 38, in main NoMacSpoofingFilter().defineNwFilter(conn) File "/usr/lib64/python2.7/site-packages/vdsm/tool/nwfilter.py", line 60, in defineNwFilter nwFilter = conn.nwfilterDefineXML(self.buildFilterXml()) File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 111, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4004, in nwfilterDefineXML if ret is None:raise libvirtError('virNWFilterDefineXML() failed', conn=self) libvirtError: operation failed: filter 'vdsm-no-mac-spoofing' already exists with uuid 10335b0e-94a2-4747-9bbb-75731d702d3a vdsm: stopped during execute nwfilter task (task returned with error code 1). Expected results: vdsmd restart is successful Additional info: To work around this problem I need to do the following command sequence: sudo virsh nwfilter-undefine vdsm-no-mac-spoofing sudo /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start sudo virsh nwfilter-undefine vdsm-no-mac-spoofing sudo service vdsmd restart