Bug 1294977 - [abrt] mock: util.py:145:rmtree:OSError: [Errno 16] Device or resource busy: '/var/lib/mock/fedora-23-x86_64/root/var/cache/dnf'
[abrt] mock: util.py:145:rmtree:OSError: [Errno 16] Device or resource busy: ...
Status: NEW
Product: Fedora
Classification: Fedora
Component: mock (Show other bugs)
26
x86_64 Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Miroslav Suchý
Fedora Extras Quality Assurance
https://retrace.fedoraproject.org/faf...
abrt_hash:cefc14d18725ff3deb992091a87...
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-31 07:16 EST by Niki Guldbrand
Modified: 2017-12-21 15:06 EST (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-12-20 12:30:58 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
File: backtrace (3.14 KB, text/plain)
2015-12-31 07:16 EST, Niki Guldbrand
no flags Details
File: environ (2.14 KB, text/plain)
2015-12-31 07:16 EST, Niki Guldbrand
no flags Details

  None (edit)
Description Niki Guldbrand 2015-12-31 07:16:37 EST
Description of problem:
An earlier mock issue had left the var/cache/dnf directory mounted in the nspawn root

Version-Release number of selected component:
mock-1.2.14-1.fc23

Additional info:
reporter:       libreport-2.6.3
cmdline:        /usr/bin/python3 -tt /sbin/mock --dnf --update --buildsrpm --spec=zsh.spec --sources=.
executable:     /sbin/mock
kernel:         4.2.8-300.fc23.x86_64
runlevel:       N 5
type:           Python3
uid:            0

Truncated backtrace:
util.py:145:rmtree:OSError: [Errno 16] Device or resource busy: '/var/lib/mock/fedora-23-x86_64/root/var/cache/dnf'

Traceback (most recent call last):
  File "/sbin/mock", line 833, in <module>
    main()
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/sbin/mock", line 649, in main
    run_command(options, args, config_opts, commands, buildroot, state)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/sbin/mock", line 732, in run_command
    do_buildsrpm(config_opts, commands, buildroot, options, args)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/sbin/mock", line 511, in do_buildsrpm
    cmd=cmd, post=None, clean=clean)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/sbin/mock", line 438, in rebuild_generic
    commands.clean()
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/backend.py", line 72, in clean
    self.buildroot.delete()
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/buildroot.py", line 563, in delete
    util.rmtree(self.basedir, selinux=self.selinux)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/util.py", line 136, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/util.py", line 136, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/util.py", line 136, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/util.py", line 136, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.4/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.4/site-packages/mockbuild/util.py", line 145, in rmtree
    os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/var/lib/mock/fedora-23-x86_64/root/var/cache/dnf'

Local variables in innermost frame:
exclude: ()
failed_to_handle: False
name: 'updates.solv'
mode: 33204
try_again: False
retries: 2
failed_filename: None
fullname: '/var/lib/mock/fedora-23-x86_64/root/var/cache/dnf/updates.solv'
names: []
path: '/var/lib/mock/fedora-23-x86_64/root/var/cache/dnf'
selinux: False
Comment 1 Niki Guldbrand 2015-12-31 07:16:46 EST
Created attachment 1110764 [details]
File: backtrace
Comment 2 Niki Guldbrand 2015-12-31 07:16:48 EST
Created attachment 1110765 [details]
File: environ
Comment 3 Robin Lee 2016-03-30 04:23:39 EDT
I met the same issue. Run mock to create a root by user1, and then try to clean the root with user2. Both user1 and user2 are mock users.
Comment 4 Mikhail Campos 2016-08-09 09:15:24 EDT
I can't reproduce the issue neither with mock-1.2.14 nor mock-1.2.18.

Do you still have the issue using mock-1.2.18?
Comment 5 Jerry James 2016-09-09 16:06:19 EDT
I just hit this issue with mock 1.2.20.  I was running mock on a remote machine, and the connection dropped during the build.  When I logged back in to the remote machine, the mock process and all build-related processes inside the chroot were gone, but /var/cache/dnf was still mounted.  As a result, I couldn't restart the build and couldn't --clean the chroot until I got a sysadmin to manually unmount that directory.

This is with mock-1.2.20-1.fc23.noarch.
Comment 6 Fedora End Of Life 2016-11-24 09:37:32 EST
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 7 Fedora End Of Life 2016-12-20 12:30:58 EST
Fedora 23 changed to end-of-life (EOL) status on 2016-12-20. Fedora 23 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.
Comment 8 Richard Kellner 2017-09-15 15:55:17 EDT
It seems I have just hit this bug as well, in Fedora 26. 

I have tried to build an rpm in mock from source rpm, and it has crashed. Unfortunately, I haven't seen the exact mock error, but I have found in journactl:

Sep 15 21:34:36 xps python3[7224]: detected unhandled Python exception in '/usr/libexec/mock/mock'

When I try to re-run the build again it ends up with an error:

INFO: mock.py version 1.4.3 starting (python version = 3.6.2)...
Start: init plugins
INFO: selinux disabled
Finish: init plugins
Start: run
INFO: Start(SRPMS/python-ludolph-weather-1.0-1.fc26.src.rpm)  Config(fedora-26-x86_64)
Start: clean chroot
ERROR: Exception(SRPMS/python-ludolph-weather-1.0-1.fc26.src.rpm) Config(fedora-26-x86_64) 0 minutes 2 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-26-x86_64/result
Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 933, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 736, in main
    run_command(options, args, config_opts, commands, buildroot, state)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 820, in run_command
    do_rebuild(config_opts, commands, buildroot, args)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 527, in do_rebuild
    post=post_build, clean=clean)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 470, in rebuild_generic
    commands.clean()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/backend.py", line 78, in clean
    self.buildroot.delete()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/buildroot.py", line 609, in delete
    util.rmtree(self.basedir, selinux=self.selinux)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 151, in rmtree
    rmtree(fullname, selinux=selinux, exclude=exclude)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/util.py", line 160, in rmtree
    os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/var/lib/mock/fedora-26-x86_64/root/var/cache/dnf'


Possibly other related information might be in this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1460522
Comment 9 Jun Aruga 2017-12-21 15:05:14 EST
Hi Richard Kellner

Which Fedora package we can reproduce this issue?
I could not find python-ludolph-weather package that you showed on Fedora.

Which version of mock did you try?

The latest version is 1.4.7.

```
$ rpm -q mock
mock-1.4.7-1.fc27.noarch
```

Note You need to log in before you can comment on or make changes to this bug.