Bug 1283628 - [RHEV-H] - setupNetworks fail sometimes with error OSError: [Errno 16] Device or resource busy
[RHEV-H] - setupNetworks fail sometimes with error OSError: [Errno 16] Device...
Status: CLOSED CURRENTRELEASE
Product: vdsm
Classification: oVirt
Component: Core (Show other bugs)
---
x86_64 All
high Severity high (vote)
: ovirt-3.6.1
: 4.17.12
Assigned To: Sagi Shnaidman
Meni Yakove
network
: Automation, Regression
Depends On:
Blocks: 1283643 1286442 1286443
  Show dependency treegraph
 
Reported: 2015-11-19 07:59 EST by Meni Yakove
Modified: 2016-02-10 14:15 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1286442 1286443 (view as bug list)
Environment:
Last Closed: 2015-12-16 07:21:20 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
sshnaidm: needinfo-
fdeutsch: ovirt‑3.6.z?
rule-engine: blocker?
myakove: testing_plan_complete+
myakove: planning_ack?
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine, vdsm and supervdsm logs (5.24 MB, application/zip)
2015-11-19 07:59 EST, Meni Yakove
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 48841 master MERGED ifcfg: remove files properly on the node Never
oVirt gerrit 49373 ovirt-3.6 MERGED ifcfg: remove files properly on the node Never
oVirt gerrit 49575 ovirt-3.5 MERGED ifcfg: remove files properly on the node Never

  None (edit)
Description Meni Yakove 2015-11-19 07:59:45 EST
Created attachment 1096671 [details]
engine, vdsm and supervdsm logs

Description of problem:
When calling setupNetwork the operation sometimes fails with error:
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,017::utils::148::root::(rmFile) Removing file: /etc/sysconfig/network-scripts/ifcfg-case4_net1 failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'
MainProcess|Thread-22381::ERROR::2015-11-18 14:44:30,018::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper
    res = func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer", line 241, in setupNetworks
    return setupNetworks(networks, bondings, **options)
  File "/usr/share/vdsm/network/api.py", line 942, in setupNetworks
    options, logger)
  File "/usr/share/vdsm/network/configurators/__init__.py", line 60, in __exit__
    leftover = self.rollback()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 83, in rollback
    self.configApplier.restoreBackups()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 466, in restoreBackups
    self.restoreAtomicBackup()
  File "/usr/share/vdsm/network/configurators/ifcfg.py", line 389, in restoreAtomicBackup
    utils.rmFile(confFilePath)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 142, in rmFile
    os.unlink(fileToRemove)
OSError: [Errno 16] Device or resource busy: '/etc/sysconfig/network-scripts/ifcfg-case4_net1'

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151112.1.el7ev)

Steps to Reproduce:
1. Run TestSanityCase13 from network sanity automation
Comment 1 Red Hat Bugzilla Rules Engine 2015-11-23 11:17:02 EST
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Comment 2 Meni Yakove 2015-11-24 07:01:32 EST
Steps to Reproduce:

1. Create VM network and attach it to host NIC
2. Change the network to be non-VM
3. Change the network to be VM network
Comment 3 Sagi Shnaidman 2015-11-29 07:38:18 EST
reproduction steps:

1. Install ovirt-node/RHEVH and connect to engine.
2. Change in the management network IP of the node to something different.

The node starts rolling back and fails because of error above.
Comment 4 Yaniv Lavi 2015-11-29 07:40:15 EST
should this be POST?
Comment 5 Sagi Shnaidman 2015-11-29 08:16:01 EST
yes
https://gerrit.ovirt.org/#/c/48841/1
Comment 6 Sandro Bonazzola 2015-12-01 10:25:45 EST
This bug is referenced in 4.17.12 git log and has target release unset.
Please check
Comment 7 Michael Burman 2015-12-06 08:39:01 EST
Verified on - 3.6.1.1-0.1.el6 with:
vdsm-4.17.12-0.el7ev.noarch
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151201.2.el7ev)
ovirt-node-3.6.0-0.23.20151201git5eed7af.el7ev.noarch
Comment 8 Sandro Bonazzola 2015-12-16 07:21:20 EST
According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.

Note You need to log in before you can comment on or make changes to this bug.