Bug 540997 - Intermittent errors creating mock root cache tarball
Summary: Intermittent errors creating mock root cache tarball
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mock
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Clark Williams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-11-24 17:10 UTC by Paul Howarth
Modified: 2013-01-10 05:37 UTC (History)
3 users (show)

Fixed In Version: 1.0.1-1.el4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-12-10 04:18:22 UTC
Type: ---


Attachments (Terms of Use)
Proposed fix for root cache creation problem (1.23 KB, patch)
2009-11-24 20:52 UTC, Clark Williams
no flags Details | Diff

Description Paul Howarth 2009-11-24 17:10:58 UTC
I have a buildsystem that targets a number of different distribution releases, and so I get to rebuild a root cache quite often. Quite frequently, the creation of the root cache tarball fails and causes the package build that triggered the root cache creation to fail. However, simply repeating the build invariably succeeds, and mock uses the supposedly failed cache tarball from the previous build without problems.

I think there are two issues.

Firstly, the cause of the tarball creation failure. Looking at the root log, it appeared to be a change in one of the files whilst it was being archived by tar.

DEBUG util.py, Line: 234:  tar:
./usr/lib/locale/locale-archive: file changed as we read it

The same problem with the same file was mentioned in a report dating back two years on fedora-devel-list:

http://www.redhat.com/archives/fedora-devel-list/2007-November/msg02599.html

More googling revealed a possible cause of the problem:
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg190963.html

So I tried forcing a "sync" before creating the tarball and lo and behold, the problem went away. I've created at least 20 root caches since making this change and all worked fine, which I'm very confident wouldn't have been the case without the "sync". So here's the change I made:

--- /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py.orig 2009-09-02 19:08:54.000000000 +0100
+++ /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py 2009-11-18 15:20:04.353035160 +0000

@@ -110,6 +110,7 @@
             # never rebuild cache unless it was a clean build.
             if self.rootObj.chrootWasCleaned:
                 self.state("creating cache")
+                mock.util.do(["sync"], shell=False)
                 mock.util.do(

["tar"] + self.compressArgs + ["-cf", self.rootCacheFile,

                      "-C", self.rootObj.makeChrootPath(), "."],


The second problem is I think that if the "tar" process to create the tarball fails (and hence causes the resulting build to fail), the cache should be invalidated so that the next build doesn't use that presumably-broken tarball. As it happens, a faulty copy of /usr/lib/locale/locale-archive doesn't seem to cause any problems during my builds but that may just be my good fortune.

Comment 1 Clark Williams 2009-11-24 20:28:21 UTC
Paul,

What do you think of this:

index 8756c01..5a83299 100644
--- a/py/mock/plugins/root_cache.py
+++ b/py/mock/plugins/root_cache.py
@@ -109,11 +109,17 @@ class RootCache(object):
             
             # never rebuild cache unless it was a clean build.
             if self.rootObj.chrootWasCleaned:
+                mock.util.do(["sync"], shell=False)
                 self.state("creating cache")
-                mock.util.do(
-                    ["tar"] + self.compressArgs + ["-cf", self.rootCacheFile,
-                     "-C", self.rootObj.makeChrootPath(), "."],
-                    shell=False
-                    )
+                try:
+                    mock.util.do(
+                        ["tar"] + self.compressArgs + ["-cf", self.rootCacheFil
+                                                       "-C", self.rootObj.makeC
+                                                       "."],
+                        shell=False
+                        )
+                except:
+                    os.remove(self.rootCacheFile)
+                    raise
         finally:
             self._rootCacheUnlock()

Comment 2 Clark Williams 2009-11-24 20:30:20 UTC
except for the stupid line-wrapping, that is :)

Intent is to just trap any exception from tar, remove the rootCacheFile and then continue the exception. Suspect I'll need to check for existance of the cache file before calling os.remove() though

Comment 3 Clark Williams 2009-11-24 20:52:31 UTC
Created attachment 373528 [details]
Proposed fix for root cache creation problem

Cleaned up version of the above inlined patch

Comment 4 Paul Howarth 2009-11-24 21:18:28 UTC
I'll have a crack at it tomorrow. If I comment out the "sync" bit I'm sure I'll be able to provoke a tar failure to test the exception hanbling.

Comment 5 Paul Howarth 2009-11-25 21:35:51 UTC
Patch works as expected. I commented out the "sync" line to provoke failures and managed to get a couple; in both cases there was no root cache tarball following the failed build. After adding the "sync" line back in, I was unable to provoke any further failures.

Comment 6 Fedora Update System 2009-11-26 05:36:01 UTC
mock-1.0.1-1.el5 has been submitted as an update for Fedora EPEL 5.
http://admin.fedoraproject.org/updates/mock-1.0.1-1.el5

Comment 7 Fedora Update System 2009-11-26 05:36:56 UTC
mock-1.0.1-1.el4 has been submitted as an update for Fedora EPEL 4.
http://admin.fedoraproject.org/updates/mock-1.0.1-1.el4

Comment 8 Fedora Update System 2009-11-26 05:37:43 UTC
mock-1.0.1-1.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc10

Comment 9 Fedora Update System 2009-11-26 05:38:07 UTC
mock-1.0.1-1.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc12

Comment 10 Fedora Update System 2009-11-26 05:38:47 UTC
mock-1.0.1-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/mock-1.0.1-1.fc11

Comment 11 Fedora Update System 2009-11-27 21:42:12 UTC
mock-1.0.1-1.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-12225

Comment 12 Fedora Update System 2009-11-27 22:01:06 UTC
mock-1.0.1-1.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-12296

Comment 13 Fedora Update System 2009-11-27 22:02:46 UTC
mock-1.0.1-1.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F12/FEDORA-2009-12306

Comment 14 Fedora Update System 2009-12-01 18:24:38 UTC
mock-1.0.1-1.el4 has been pushed to the Fedora EPEL 4 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/EL-4/FEDORA-EPEL-2009-0909

Comment 15 Fedora Update System 2009-12-01 18:27:12 UTC
mock-1.0.1-1.el5 has been pushed to the Fedora EPEL 5 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/EL-5/FEDORA-EPEL-2009-0928

Comment 16 Fedora Update System 2009-12-10 04:18:16 UTC
mock-1.0.1-1.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2009-12-10 04:25:59 UTC
mock-1.0.1-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2009-12-17 23:53:44 UTC
mock-1.0.1-1.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2009-12-17 23:56:06 UTC
mock-1.0.1-1.el4 has been pushed to the Fedora EPEL 4 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.