Bug 1493244

Summary: Cleanup failed; device is busy
Product: [Community] Copr Reporter: Vincent Batts <vbatts>
Component: backendAssignee: Miroslav Suchý <msuchy>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: clime, praiskup
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-06 15:02:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vincent Batts 2017-09-19 17:41:20 UTC
Description of problem:
https://copr.fedorainfracloud.org/coprs/vbatts/envoy/build/604247/
https://copr-be.cloud.fedoraproject.org/results/vbatts/envoy/epel-7-x86_64/00604247-envoy/builder-live.log
end of that log shows a successful build, but failed cleanup:
```
+ cd /builddir/build/BUILD
+ cd envoy-ec3446e9b3631568db6223e9eaf37fc6ce9f6cf2
+ /usr/bin/rm -rf /builddir/build/BUILDROOT/envoy-1.4.0.1.git.ec3446e-1.el7.centos.x86_64
+ exit 0
Finish: rpmbuild envoy-1.4.0.1.git.ec3446e-1.fc26.src.rpm
Finish: build phase for envoy-1.4.0.1.git.ec3446e-1.fc26.src.rpm
INFO: Done(/tmp/tmp6bczse2a/repo/envoy-1.4.0.1.git.ec3446e-1.fc26.src.rpm) Config(child) 22 minutes 29 seconds
INFO: Results and/or logs in: /var/lib/copr-rpmbuild/results
INFO: Cleaning up build root ('cleanup_on_success=True')
Start: clean chroot
WARNING: Process ID 6527 still running in chroot. Killing...
INFO: unmounting tmpfs.
WARNING: tmpfs-plugin: exception while umounting tmpfs! (cwd: /home/mockbuilder)
WARNING: tmpfs-plugin: exception while force umounting tmpfs! (cwd: /home/mockbuilder)
ERROR: Exception(/tmp/tmp6bczse2a/repo/envoy-1.4.0.1.git.ec3446e-1.fc26.src.rpm) Config(604247-epel-7-x86_64) 22 minutes 46 seconds
INFO: Results and/or logs in: /var/lib/copr-rpmbuild/results
INFO: Cleaning up build root ('cleanup_on_failure=True')
Start: clean chroot
Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 480, in rebuild_generic
    commands.clean()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/backend.py", line 78, in clean
    self.buildroot.delete()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/buildroot.py", line 609, in delete
    util.rmtree(self.basedir, selinux=self.selinux)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 160, in rmtree
    os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/var/lib/mock/604247-epel-7-x86_64/root'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 933, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 736, in main
    run_command(options, args, config_opts, commands, buildroot, state)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 820, in run_command
    do_rebuild(config_opts, commands, buildroot, args)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 527, in do_rebuild
    post=post_build, clean=clean)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 492, in rebuild_generic
    commands.clean()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/backend.py", line 78, in clean
    self.buildroot.delete()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/buildroot.py", line 609, in delete
    util.rmtree(self.basedir, selinux=self.selinux)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 160, in rmtree
    os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/var/lib/mock/604247-epel-7-x86_64/root'
Build failed
```


Version-Release number of selected component (if applicable):


How reproducible:
inconsistent

Steps to Reproduce:
1. build triggered from github webhook
2.
3.

Actual results:
some of the fedora or centos builds are failing with device-busy during cleanup

Expected results:
successful build, regardless of the cleanup task.

Additional info:

Comment 1 clime 2017-09-21 07:39:48 UTC
This is rather an issue in mock: 

Product: 	Fedora
Component: 	mock

I would recommend opening the bug there as well and put link here to track it.

Comment 2 clime 2018-04-06 15:02:37 UTC
The problem in mock is fixed now.