Bug 1776956 - [abrt] mock: logOutput(): util.py:265:rmtree:OSError: [Errno 16] Device or resource busy: '/var/lib/mock/fedora-rawhide-i386/root/proc/filesystems'
Summary: [abrt] mock: logOutput(): util.py:265:rmtree:OSError: [Errno 16] Device or re...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mock
Version: 32
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Copr Team
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:6ca0ddb74edde114335136abbfc...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-26 16:16 UTC by Mikhail
Modified: 2020-04-20 22:31 UTC (History)
8 users (show)

Fixed In Version: mock-2.2-1.fc32 mock-2.2-1.fc31 mock-2.2-1.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-04-07 05:05:04 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (5.13 KB, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details
File: cpuinfo (2.38 KB, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details
File: environ (204 bytes, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details
File: mountinfo (2.92 KB, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details
File: namespaces (121 bytes, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details
File: open_fds (1.01 KB, text/plain)
2019-11-26 16:16 UTC, Mikhail
no flags Details

Description Mikhail 2019-11-26 16:16:37 UTC
Version-Release number of selected component:
mock-1.4.21-1.fc32

Additional info:
reporter:       libreport-2.11.3
cgroup:         0::/user.slice/user-1000.slice/user/gnome-terminal-server.service
cmdline:        /usr/bin/python3 -tt /usr/libexec/mock/mock -r fedora-rawhide-i386 --resultdir /home/mikhail/packaging-work/mesa/results_mesa/19.3.0~rc4/1.fc32 --rebuild /home/mikhail/packaging-work/mesa/mesa-19.3.0~rc4-1.fc32.src.rpm
crash_function: logOutput
exception_type: KeyboardInterrupt
executable:     /usr/libexec/mock/mock
interpreter:    python3-3.8.0-1.fc32.x86_64
kernel:         5.4.0-0.rc8.git0.2.fc32.x86_64
runlevel:       N 5
type:           Python3
uid:            0

Truncated backtrace:
#1 [/usr/lib/python3.8/site-packages/mockbuild/util.py:551] logOutput
#2 [/usr/lib/python3.8/site-packages/mockbuild/util.py:705] do_with_status
#3 [/usr/lib/python3.8/site-packages/mockbuild/trace_decorator.py:95] trace
#4 [/usr/lib/python3.8/site-packages/mockbuild/util.py:652] do
#5 [/usr/lib/python3.8/site-packages/mockbuild/plugins/selinux.py:106] _selinuxDoYum
#6 [/usr/lib/python3.8/site-packages/mockbuild/trace_decorator.py:95] trace
#7 [/usr/lib/python3.8/site-packages/mockbuild/package_manager.py:154] execute
#8 [/usr/lib/python3.8/site-packages/mockbuild/trace_decorator.py:95] trace
#9 [/usr/lib/python3.8/site-packages/mockbuild/package_manager.py:195] builddep
#10 [/usr/lib/python3.8/site-packages/mockbuild/trace_decorator.py:95] trace

Comment 1 Mikhail 2019-11-26 16:16:41 UTC
Created attachment 1639906 [details]
File: backtrace

Comment 2 Mikhail 2019-11-26 16:16:43 UTC
Created attachment 1639907 [details]
File: cpuinfo

Comment 3 Mikhail 2019-11-26 16:16:44 UTC
Created attachment 1639908 [details]
File: environ

Comment 4 Mikhail 2019-11-26 16:16:46 UTC
Created attachment 1639909 [details]
File: mountinfo

Comment 5 Mikhail 2019-11-26 16:16:47 UTC
Created attachment 1639910 [details]
File: namespaces

Comment 6 Mikhail 2019-11-26 16:16:49 UTC
Created attachment 1639911 [details]
File: open_fds

Comment 7 Jerry James 2020-01-18 23:04:56 UTC
To reproduce this, simply start a mock build, wait a couple of seconds, then hit Control-C.  Now try to start the mock build again and you get this error.  The user cannot fix it.  Mock doesn't clean up the problem itself, nor does mock --clean, and the user doesn't have permission to unmount the problematic filesystem.  Only root can unmount the filesystem so that the buildroot is usable again.

Note that recently, I've found myself needing to sudo umount *two* filesystems:

<mock-root>/root/proc/filesystems
<mock-root>/root/var/cache/dnf

It would be great if mock --clean, at least, could do that.

Comment 8 Ben Cotton 2020-02-11 17:33:22 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 9 Pavel Raiskup 2020-04-02 11:58:18 UTC
I believe those two, especially the second ...

https://github.com/rpm-software-management/mock/commit/2ecc9071df3cf2ceb10765c65abc8a8d82414b57
https://github.com/rpm-software-management/mock/commit/2185a51110e7129c90b61ac3abd7cefcd6c458da

... fixed these kind of issues.

My explanation would that

-> we killed `dnf install` command in the middle of the transaction
-> some background process was kept working hard on background
-> the process had opened some file descriptors that refused mock to
   umount all the mountpoins
-> mock failed hard, and
-> the mount layout ended up in inconsistent state enough so even
   further `mock --scrub=all` failed.

I'm attaching this into 2.2 bodhi update.  Please reopen if you see the
same problems with mock 2.2+.

Comment 10 Fedora Update System 2020-04-02 11:59:24 UTC
FEDORA-2020-fba9845e22 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

Comment 11 Fedora Update System 2020-04-02 11:59:49 UTC
FEDORA-2020-6b7c342fb4 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

Comment 12 Fedora Update System 2020-04-02 12:01:02 UTC
FEDORA-2020-85df0014c1 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

Comment 13 Fedora Update System 2020-04-03 19:56:06 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-fba9845e22`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2020-04-03 20:35:06 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-85df0014c1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 15 Fedora Update System 2020-04-03 20:49:35 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-6b7c342fb4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2020-04-07 05:05:04 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2020-04-07 13:16:01 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 Fedora Update System 2020-04-20 22:14:47 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 19 Fedora Update System 2020-04-20 22:31:59 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.