I have a buildsystem that targets a number of different distribution releases, and so I get to rebuild a root cache quite often. Quite frequently, the creation of the root cache tarball fails and causes the package build that triggered the root cache creation to fail. However, simply repeating the build invariably succeeds, and mock uses the supposedly failed cache tarball from the previous build without problems. I think there are two issues. Firstly, the cause of the tarball creation failure. Looking at the root log, it appeared to be a change in one of the files whilst it was being archived by tar. DEBUG util.py, Line: 234: tar: ./usr/lib/locale/locale-archive: file changed as we read it The same problem with the same file was mentioned in a report dating back two years on fedora-devel-list: http://www.redhat.com/archives/fedora-devel-list/2007-November/msg02599.html More googling revealed a possible cause of the problem: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg190963.html So I tried forcing a "sync" before creating the tarball and lo and behold, the problem went away. I've created at least 20 root caches since making this change and all worked fine, which I'm very confident wouldn't have been the case without the "sync". So here's the change I made: --- /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py.orig 2009-09-02 19:08:54.000000000 +0100 +++ /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py 2009-11-18 15:20:04.353035160 +0000 @@ -110,6 +110,7 @@ # never rebuild cache unless it was a clean build. if self.rootObj.chrootWasCleaned: self.state("creating cache") + mock.util.do(["sync"], shell=False) mock.util.do( ["tar"] + self.compressArgs + ["-cf", self.rootCacheFile, "-C", self.rootObj.makeChrootPath(), "."], The second problem is I think that if the "tar" process to create the tarball fails (and hence causes the resulting build to fail), the cache should be invalidated so that the next build doesn't use that presumably-broken tarball. As it happens, a faulty copy of /usr/lib/locale/locale-archive doesn't seem to cause any problems during my builds but that may just be my good fortune.
Paul, What do you think of this: index 8756c01..5a83299 100644 --- a/py/mock/plugins/root_cache.py +++ b/py/mock/plugins/root_cache.py @@ -109,11 +109,17 @@ class RootCache(object): # never rebuild cache unless it was a clean build. if self.rootObj.chrootWasCleaned: + mock.util.do(["sync"], shell=False) self.state("creating cache") - mock.util.do( - ["tar"] + self.compressArgs + ["-cf", self.rootCacheFile, - "-C", self.rootObj.makeChrootPath(), "."], - shell=False - ) + try: + mock.util.do( + ["tar"] + self.compressArgs + ["-cf", self.rootCacheFil + "-C", self.rootObj.makeC + "."], + shell=False + ) + except: + os.remove(self.rootCacheFile) + raise finally: self._rootCacheUnlock()
except for the stupid line-wrapping, that is :) Intent is to just trap any exception from tar, remove the rootCacheFile and then continue the exception. Suspect I'll need to check for existance of the cache file before calling os.remove() though
Created attachment 373528 [details] Proposed fix for root cache creation problem Cleaned up version of the above inlined patch
I'll have a crack at it tomorrow. If I comment out the "sync" bit I'm sure I'll be able to provoke a tar failure to test the exception hanbling.
Patch works as expected. I commented out the "sync" line to provoke failures and managed to get a couple; in both cases there was no root cache tarball following the failed build. After adding the "sync" line back in, I was unable to provoke any further failures.
mock-1.0.1-1.el5 has been submitted as an update for Fedora EPEL 5. http://admin.fedoraproject.org/updates/mock-1.0.1-1.el5
mock-1.0.1-1.el4 has been submitted as an update for Fedora EPEL 4. http://admin.fedoraproject.org/updates/mock-1.0.1-1.el4
mock-1.0.1-1.fc10 has been submitted as an update for Fedora 10. http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc10
mock-1.0.1-1.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc12
mock-1.0.1-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc11
mock-1.0.1-1.fc10 has been pushed to the Fedora 10 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update mock'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-12225
mock-1.0.1-1.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update mock'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-12296
mock-1.0.1-1.fc12 has been pushed to the Fedora 12 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update mock'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F12/FEDORA-2009-12306
mock-1.0.1-1.el4 has been pushed to the Fedora EPEL 4 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update mock'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/EL-4/FEDORA-EPEL-2009-0909
mock-1.0.1-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update mock'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/EL-5/FEDORA-EPEL-2009-0928
mock-1.0.1-1.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.0.1-1.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.0.1-1.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.0.1-1.el4 has been pushed to the Fedora EPEL 4 stable repository. If problems still persist, please make note of it in this bug report.