Bug 1330636

Summary: anaconda failing during some livemedia builds
Product: [Fedora] Fedora Reporter: Bruno Wolff III <bruno>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 24CC: anaconda-maint-list, bruno, g.kaviyarasu, jonathan, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
URL: https://kojipkgs.fedoraproject.org//work/tasks/7906/13807906/livemedia-out.log
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-27 20:40:38 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Bruno Wolff III 2016-04-26 15:52:11 UTC
User-Agent:       
Build Identifier: 

The cause of the error isn't obvious from the messages in the logs (at least to me). It doesn't look like the problem is with the spin or livemedia-creator, but it is hard to tell for sure.

Reproducible: Sometimes

Actual Results:  
2016-04-26 10:34:07,296 INFO livemedia-creator: Running post-installation scripts
2016-04-26 10:34:07,300 INFO livemedia-creator: .
2016-04-26 10:34:08,542 INFO livemedia-creator: Error in atexit._run_exitfuncs:
2016-04-26 10:34:08,542 INFO livemedia-creator: Traceback (most recent call last):
2016-04-26 10:34:08,543 INFO livemedia-creator: File "/usr/sbin/anaconda", line 80, in exitHandler
2016-04-26 10:34:08,543 INFO livemedia-creator: anaconda.storage.umountFilesystems(swapoff=False)
2016-04-26 10:34:08,543 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/blivet.py", line 1410, in umountFilesystems
2016-04-26 10:34:08,543 INFO livemedia-creator: self.fsset.umountFilesystems(swapoff=swapoff)
2016-04-26 10:34:08,543 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/osinstall.py", line 655, in umountFilesystems
2016-04-26 10:34:08,543 INFO livemedia-creator: device.format.teardown()
2016-04-26 10:34:08,544 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/formats/__init__.py", line 526, in teardown
2016-04-26 10:34:08,544 INFO livemedia-creator: self._teardown(**kwargs)
2016-04-26 10:34:08,544 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/formats/fs.py", line 698, in _teardown
2016-04-26 10:34:08,544 INFO livemedia-creator: raise FSError("umount of %s failed (%d)" % (mountpoint, rc))
2016-04-26 10:34:08,544 INFO livemedia-creator: blivet.errors.FSError: umount of /mnt/sysimage/dev failed (32)
2016-04-26 10:35:41,272 INFO livemedia-creator: Shutting down log processing
2016-04-26 10:35:48,346 INFO livemedia-creator: Disk Image install successful
2016-04-26 10:35:48,347 INFO livemedia-creator: working dir is /var/tmp/lmc-work-kqxhsb23
2016-04-26 10:35:48,731 ERROR livemedia-creator: Problem zeroing free blocks of /tmp/lmc/lmc-disk-5bwwh2wz.img
2016-04-26 10:35:48,732 ERROR livemedia-creator: squashfs.img creation failed

Comment 1 Brian Lane 2016-04-26 23:57:17 UTC
Are you seeing this running locally or via koji? See bug 1315541

Something is interfereing with unmounting, resulting in a disk image with a corrupt journal. The ext4 discard pass catches this and raises the error you are seeing. The actual cause is something else (and is currently unknown).

I've been unable to reproduce it locally, so if you are maybe you can figure out what's going on.

Comment 2 Bruno Wolff III 2016-04-27 02:30:38 UTC
I saw it happen on the nightly build of the games spin in koji. It doesn't happen all of the time. I think it may also be happening to some other spins, but I haven't looked at them closely.

I saw the referenced bug, but I don't know whether or not it is related.

It seems like it might be hitting the games spin more than other spins (or labs as they now seem to be called). Games seems to fail to build more often than the others.

Comment 3 Bruno Wolff III 2016-04-27 11:18:35 UTC
Today the affected file system is different than yesterday:
2016-04-27 10:18:16,648 INFO livemedia-creator: Running post-installation scripts
2016-04-27 10:18:16,653 INFO livemedia-creator: .
2016-04-27 10:18:17,173 INFO livemedia-creator: Error in atexit._run_exitfuncs:
2016-04-27 10:18:17,173 INFO livemedia-creator: Traceback (most recent call last):
2016-04-27 10:18:17,173 INFO livemedia-creator: File "/usr/sbin/anaconda", line 80, in exitHandler
2016-04-27 10:18:17,173 INFO livemedia-creator: anaconda.storage.umountFilesystems(swapoff=False)
2016-04-27 10:18:17,173 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/blivet.py", line 1410, in umountFilesystems
2016-04-27 10:18:17,174 INFO livemedia-creator: self.fsset.umountFilesystems(swapoff=swapoff)
2016-04-27 10:18:17,174 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/osinstall.py", line 655, in umountFilesystems
2016-04-27 10:18:17,174 INFO livemedia-creator: device.format.teardown()
2016-04-27 10:18:17,174 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/formats/__init__.py", line 526, in teardown
2016-04-27 10:18:17,174 INFO livemedia-creator: self._teardown(**kwargs)
2016-04-27 10:18:17,175 INFO livemedia-creator: File "/usr/lib/python3.5/site-packages/blivet/formats/fs.py", line 698, in _teardown
2016-04-27 10:18:17,175 INFO livemedia-creator: raise FSError("umount of %s failed (%d)" % (mountpoint, rc))
2016-04-27 10:18:17,175 INFO livemedia-creator: blivet.errors.FSError: umount of /mnt/sysimage/dev/shm failed (32)
2016-04-27 10:19:55,569 INFO livemedia-creator: Shutting down log processing
2016-04-27 10:19:58,444 INFO livemedia-creator: Disk Image install successful
2016-04-27 10:19:58,445 INFO livemedia-creator: working dir is /var/tmp/lmc-work-9v4pstr4
2016-04-27 10:19:58,576 ERROR livemedia-creator: Problem zeroing free blocks of /tmp/lmc/lmc-disk-p37xnan9.img
2016-04-27 10:19:58,576 ERROR livemedia-creator: squashfs.img creation failed

Comment 4 Brian Lane 2016-04-27 20:40:38 UTC
Darn, I was hoping you could reproduce it locally. This is a dupe and it is hitting all of them randomly as far as I can tell.

*** This bug has been marked as a duplicate of bug 1315541 ***