Bug 1283628 - [RHEV-H] - setupNetworks fail sometimes with error OSError: [Errno 16] Device or resource busy
Summary: [RHEV-H] - setupNetworks fail sometimes with error OSError: [Errno 16] Device...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: ---
Hardware: x86_64
OS: All
high
high
Target Milestone: ovirt-3.6.1
: 4.17.12
Assignee: Sagi Shnaidman
QA Contact: Meni Yakove
URL:
Whiteboard: network
Depends On:
Blocks: 1283643 1286442 1286443
TreeView+ depends on / blocked
 
Reported: 2015-11-19 12:59 UTC by Meni Yakove
Modified: 2016-02-10 19:15 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1286442 1286443 (view as bug list)
Environment:
Last Closed: 2015-12-16 12:21:20 UTC
oVirt Team: Network
Embargoed:
sshnaidm: needinfo-
fdeutsch: ovirt-3.6.z?
rule-engine: blocker?
myakove: testing_plan_complete+
myakove: planning_ack?
rule-engine: devel_ack+
rule-engine: testing_ack+

Attachments (Terms of Use)
engine, vdsm and supervdsm logs (5.24 MB, application/zip)
2015-11-19 12:59 UTC, Meni Yakove 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 48841 0 master MERGED ifcfg: remove files properly on the node Never
oVirt gerrit 49373 0 ovirt-3.6 MERGED ifcfg: remove files properly on the node Never
oVirt gerrit 49575 0 ovirt-3.5 MERGED ifcfg: remove files properly on the node Never

Description Meni Yakove 2015-11-19 12:59:45 UTC 
Created attachment 1096671 [details]
engine, vdsm and supervdsm logs

Description of problem:
When calling setupNetwork the operation sometimes fails with error:
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,017::utils::148::root::(rmFile) Removing file: /etc/sysconfig/network-scripts/ifcfg-case4_net1 failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,018::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper
    res = func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer", line 241, in setupNetworks
    return setupNetworks(networks, bondings, **options)
  File "/usr/share/vdsm/network/api.py", line 942, in setupNetworks
    options, logger)
  File "/usr/share/vdsm/network/configurators/__init__.py", line 60, in __exit__
    leftover = self.rollback()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 83, in rollback
    self.configApplier.restoreBackups()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 466, in restoreBackups
    self.restoreAtomicBackup()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 389, in restoreAtomicBackup
    utils.rmFile(confFilePath)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151112.1.el7ev)

Steps to Reproduce:
1. Run TestSanityCase13 from network sanity automation
Comment 1 Red Hat Bugzilla Rules Engine 2015-11-23 16:17:02 UTC 
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Comment 2 Meni Yakove 2015-11-24 12:01:32 UTC 
Steps to Reproduce:

1. Create VM network and attach it to host NIC
2. Change the network to be non-VM
3. Change the network to be VM network
Comment 3 Sagi Shnaidman 2015-11-29 12:38:18 UTC 
reproduction steps:

1. Install ovirt-node/RHEVH and connect to engine.
2. Change in the management network IP of the node to something different.

The node starts rolling back and fails because of error above.
Comment 4 Yaniv Lavi 2015-11-29 12:40:15 UTC 
should this be POST?
Comment 5 Sagi Shnaidman 2015-11-29 13:16:01 UTC 
yes
https://gerrit.ovirt.org/#/c/48841/1
Comment 6 Sandro Bonazzola 2015-12-01 15:25:45 UTC 
This bug is referenced in 4.17.12 git log and has target release unset.
Please check
Comment 7 Michael Burman 2015-12-06 13:39:01 UTC 
Verified on - 3.6.1.1-0.1.el6 with:
vdsm-4.17.12-0.el7ev.noarch
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151201.2.el7ev)
ovirt-node-3.6.0-0.23.20151201git5eed7af.el7ev.noarch
Comment 8 Sandro Bonazzola 2015-12-16 12:21:20 UTC 
According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.
Note You need to log in before you can comment on or make changes to this bug.