|Summary:||workaround for for appliance creation in koji|
|Product:||[Fedora] Fedora||Reporter:||Matthew Miller <mattdm>|
|Component:||device-mapper-multipath||Assignee:||Ben Marzinski <bmarzins>|
|Status:||CLOSED CURRENTRELEASE||QA Contact:||Fedora Extras Quality Assurance <extras-qa>|
|Version:||18||CC:||agk, agrimm, apevec+fedora, awilliam, bmarzins, dhuff, dwysocha, fdinitto, gholms, heinzm, lvm-team, mgoldman, msnitzer, prajnoha, prockai, robatino, virt-maint|
|Fixed In Version:||device-mapper-multipath-0.4.9-36.fc18||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2013-01-02 21:51:32 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
|Bug Depends On:|
Description Matthew Miller 2012-12-14 19:24:20 UTC
There's a recent new problem in F18 which causes appliance creation to fail in the koji buildsystem, because loopback devices aren't created properly inside the chroot. Andy Grimm has created a patch http://fpaste.org/LwiV/ (also, follows) which we'd like to apply to work around the issue. There may be a deeper issue with device-mapper that is the root cause, but this is a small and relatively unobtrusive fix which should get us back to building images without much implication. Also nominating as F18 blocker, as functioning EC2 images is a final release criteria and we'll need this and it can't be an update. ---- --- appliance-tools-006.2.orig/appcreate/partitionedfs.py 2012-08-09 03:29:33.000000000 -0700 +++ appliance-tools-006.2.orig/appcreate/partitionedfs.py 2012-12-14 10:36:57.982845602 -0800 @@ -173,6 +173,21 @@ if rc != 0: raise MountError("Failed to map partitions for '%s'" % d['disk'].device) + if not os.path.exists(mapperdev): + dmsetup = subprocess.Popen(["dmsetup", "ls"], + stdout=subprocess.PIPE) + dmsetupOutput = dmsetup.communicate().split('\n') + for line in dmsetupOutput: + m = re.match("%s\s+\((\d+):(\d+)\)" % os.path.basename(mapperdev), line) + if not m: + continue + major, minor = m.groups()[:2] + rc = subprocess.call(["/bin/mknod", mapperdev, "b", major, minor]) + if rc != 0: + raise MountError("Failed to map partitions for '%s'" % + d['disk'].device) + break + d['mapped'] = True
Comment 1 Andy Grimm 2012-12-14 20:25:27 UTC
Upon further review, appliance-creator tasks in koji do bind mount /dev (thanks dgilmore for pointing that out). Given that, we are back to this being a race condition. I've verified that I can reproduce it, and sleeping even a fraction of a second (0.1 or less) is enough for the problem to go away. I've also verified that in doing os.listdir("/dev/mapper") immediately after the kpartx call returns, the /dev/mapper symlink is not present yet. So this is the same issue as 885385 after all. Sorry for the confusion.
Comment 2 Andy Grimm 2012-12-14 21:47:25 UTC
Let's clarify one more time. It's a race, but not the same race as 885385 This one is because kpartx's -s option doesn't work when udev is not running in the chroot.
Comment 3 Ben Marzinski 2012-12-14 22:46:54 UTC
device-mapper-multipath packages that reinstate the libdm failback device creation for multipath and kpartx when called in sync mode are available at: http://koji.fedoraproject.org/koji/taskinfo?taskID=4792365 Let me know if this fixes the problem.
Comment 4 Andy Grimm 2012-12-15 00:05:23 UTC
Confirmed that the above build fixes the problem. Thanks, Ben.
Comment 5 Adam Williamson 2012-12-17 19:01:16 UTC
Discussed at 2012-12-17 blocker review meeting: http://meetbot.fedoraproject.org/fedora-bugzappers/2012-12-17/f18final-blocker-review-5.2012-12-17-16.40.log.txt . Accepted as a blocker per criterion "The release must boot successfully as Xen DomU with releases providing a functional, supported Xen Dom0 and widely used cloud providers utilizing Xen. This does not include any issues limited to the release functioning as Xen Dom0" - this bug blocks creation of the EC2 image, which is a major case of that criterion.
Comment 6 Ben Marzinski 2012-12-17 19:52:19 UTC
fix added. multipath and kpartx now use the libdm failback device creation when called in sync mode.
Comment 7 Fedora Update System 2012-12-17 20:13:44 UTC
device-mapper-multipath-0.4.9-36.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/device-mapper-multipath-0.4.9-36.fc18
Comment 8 Fedora Update System 2012-12-18 21:26:41 UTC
Package device-mapper-multipath-0.4.9-36.fc18: * should fix your issue, * was pushed to the Fedora 18 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing device-mapper-multipath-0.4.9-36.fc18' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-20573/device-mapper-multipath-0.4.9-36.fc18 then log in and leave karma (feedback).
Comment 9 Fedora Update System 2013-01-02 21:51:35 UTC
device-mapper-multipath-0.4.9-36.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.