Bug 1210569 - Race condition in mock's ccache plugin
Summary: Race condition in mock's ccache plugin
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mock
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miroslav Suchý
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-10 06:02 UTC by Jeff Law
Modified: 2015-06-20 21:13 UTC (History)
6 users (show)

Fixed In Version: mock-1.2.10-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-05-01 16:50:51 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jeff Law 2015-04-10 06:02:06 UTC
Description of problem:
mock instances can share a ccache directory by using the --uniqueext option and a shared mock configuration file.

It is possible for one mock instance to be removing files from the ccache (for example if the cache fills up, or the various lock files used internally by ccache) while the other instance is trying to ensure consistent ownership of files in the cache via this code in ccache.py:

        self.buildroot.uid_manager.changeOwner(self.ccachePath, recursive=True)


If we look at the implementation of changeOwner we have:

        if recursive:
            for root, dirs, files in os.walk(path):
                for d in dirs:
                    os.chown(os.path.join(root, d), uid, gid)
                for f in files:
                    os.chown(os.path.join(root, f), uid, gid)


Presumably os.walk (path) enumerates all the files and directories, then the loop over them proceeds to change their ownership.  Obviously if a file is removed from the cache between the enumeration step and the actual chown call we'll get an error like this:

  File "/usr/lib/python2.7/site-packages/mockbuild/uid.py", line 84, in changeOwner
    os.chown(os.path.join(root, f), uid, gid)
OSError: [Errno 2] No such file or directory: '/var/cache/mock/epel-7-x86_64/ccache/u0/2/stats.lock'

Of course this aborts the mock build.

Version-Release number of selected component (if applicable):


How reproducible:
Keep many mock instances building SRPMS running which share a configuration file and use --uniqueext.  It's probably easier to trigger if the ccache size is kept small (which will encourage eviction of files from the cache).  Watch for error messages similar to the one above.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Miroslav Suchý 2015-04-22 07:12:59 UTC
Fixed in:
* b65ed15  ignore missing files in ccache [RHBZ#1210569]

Comment 2 Fedora Update System 2015-04-29 13:16:36 UTC
mock-1.2.8-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/mock-1.2.8-1.fc22

Comment 3 Fedora Update System 2015-04-29 13:17:01 UTC
mock-1.2.8-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/mock-1.2.8-1.fc21

Comment 4 Fedora Update System 2015-04-29 13:17:49 UTC
mock-1.2.8-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/mock-1.2.8-1.fc20

Comment 5 Fedora Update System 2015-04-29 13:19:04 UTC
mock-1.2.8-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/mock-1.2.8-1.el7

Comment 6 Fedora Update System 2015-04-29 13:25:49 UTC
mock-1.2.8-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.2.8-1.el6

Comment 7 Fedora Update System 2015-04-29 19:18:46 UTC
Package mock-1.2.8-1.el7:
* should fix your issue,
* was pushed to the Fedora EPEL 7 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing mock-1.2.8-1.el7'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2015-6057/mock-1.2.8-1.el7
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2015-05-01 16:50:51 UTC
mock-1.2.8-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2015-05-02 18:08:29 UTC
mock-1.2.8-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2015-05-12 20:46:29 UTC
mock-1.2.8-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2015-06-04 11:42:16 UTC
mock-1.2.10-1.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/mock-1.2.10-1.el7

Comment 12 Fedora Update System 2015-06-04 11:42:57 UTC
mock-1.2.10-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.2.10-1.el6

Comment 13 Fedora Update System 2015-06-20 21:10:38 UTC
mock-1.2.10-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2015-06-20 21:13:34 UTC
mock-1.2.10-1.el7 has been pushed to the Fedora EPEL 7 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.