Bug 1286442

Summary:	[RHEV-H] - setupNetworks fail sometimes with error OSError: [Errno 16] Device or resource busy
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Sagi Shnaidman <sshnaidm>
Component:	vdsm	Assignee:	Sagi Shnaidman <sshnaidm>
Status:	CLOSED ERRATA	QA Contact:	Meni Yakove <myakove>
Severity:	high	Docs Contact:
Priority:	high
Version:	2.1.0	CC:	bazulay, bugs, danken, ecohen, eedri, fdeutsch, gklein, lsurette, mburman, myakove, sshnaidm, ycui, yeylon, ylavi
Target Milestone:	ovirt-3.5.7	Keywords:	Automation, Regression, ZStream
Target Release:	3.5.7
Hardware:	x86_64
OS:	All
Whiteboard:	network
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1283628	Environment:
Last Closed:	2016-01-12 20:35:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Network	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1283628, 1286443, 1293617, 1294075
Bug Blocks:	1283643

Description Sagi Shnaidman 2015-11-29 14:46:37 UTC

+++ This bug was initially created as a clone of Bug #1283628 +++

Description of problem:
When calling setupNetwork the operation sometimes fails with error:
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,017::utils::148::root::(rmFile) Removing file: /etc/sysconfig/network-scripts/ifcfg-case4_net1 failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,018::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper
    res = func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer", line 241, in setupNetworks
    return setupNetworks(networks, bondings, **options)
  File "/usr/share/vdsm/network/api.py", line 942, in setupNetworks
    options, logger)
  File "/usr/share/vdsm/network/configurators/__init__.py", line 60, in __exit__
    leftover = self.rollback()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 83, in rollback
    self.configApplier.restoreBackups()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 466, in restoreBackups
    self.restoreAtomicBackup()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 389, in restoreAtomicBackup
    utils.rmFile(confFilePath)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151112.1.el7ev)

Steps to Reproduce:
1. Run TestSanityCase13 from network sanity automation

--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-11-23 11:17:02 EST ---

This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

--- Additional comment from Meni Yakove on 2015-11-24 07:01:32 EST ---

Steps to Reproduce:

1. Create VM network and attach it to host NIC
2. Change the network to be non-VM
3. Change the network to be VM network

--- Additional comment from Sagi Shnaidman on 2015-11-29 07:38:18 EST ---

reproduction steps:

1. Install ovirt-node/RHEVH and connect to engine.
2. Change in the management network IP of the node to something different.

The node starts rolling back and fails because of error above.

--- Additional comment from Yaniv Dary on 2015-11-29 07:40:15 EST ---

should this be POST?

--- Additional comment from Sagi Shnaidman on 2015-11-29 08:16:01 EST ---

yes
https://gerrit.ovirt.org/#/c/48841/1

Comment 1 Red Hat Bugzilla Rules Engine 2015-11-29 14:46:41 UTC

This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.

Comment 2 Red Hat Bugzilla Rules Engine 2015-11-29 15:00:30 UTC

This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.

Comment 3 Sagi Shnaidman 2015-11-29 15:02:18 UTC

*** Bug 1286443 has been marked as a duplicate of this bug. ***

Comment 4 Red Hat Bugzilla Rules Engine 2015-11-29 15:10:36 UTC

This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 6 Michael Burman 2015-12-30 15:20:54 UTC

Verified on - 3.5.7-0.1.el6ev
vdsm-4.16.32-1.el7ev.x86_64
ovirt-node-3.2.3-30.el7.noarch
Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151229.1.el7ev)

Comment 8 errata-xmlrpc 2016-01-12 20:35:08 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0034.html