Bug 1650690

Summary:	openstack overcloud image build fails with ERROR: failed to umount the /tmp/dib_image.e48mtbV2 tmpfs mount point
Product:	Red Hat OpenStack	Reporter:	Alexander Chuzhoy <sasha>
Component:	diskimage-builder	Assignee:	Yolanda Robla <yroblamo>
Status:	CLOSED DUPLICATE	QA Contact:	Arik Chernetsky <achernet>
Severity:	high	Docs Contact:
Priority:	high
Version:	13.0 (Queens)	CC:	bfournie, ltoscano, mburns
Target Milestone:	---	Keywords:	AutomationBlocker, TestBlocker, ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-11-26 18:40:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2018-11-16 20:58:32 UTC

openstack overcloud image build fails with ERROR: failed to umount the /tmp/dib_image.e48mtbV2 tmpfs mount point


Environment:
diskimage-builder-2.16.0-2.el7ost.noarch
instack-undercloud-8.4.6-2.el7ost.noarch
rhel-guest-image-7.6-215.x86_64.qcow2


Steps to reproduce:


1. Deploy OSP13 undercloud
2. Download image rhel-guest-image-7.6-215.x86_64.qcow2
3. Run w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1646976:

LIBGUESTFS_BACKEND=direct virt-edit -a ./rhel-guest-image-7.6-215.x86_64.qcow2 /etc/default/grub -e 's/(GRUB_CMDLINE_LINUX="console=tty0).*/$1,115200n8 crashkernel=auto no_timer_check net.ifnames=0"/'


virt-cat -a ./rhel-guest-image-7.6-215.x86_64.qcow2 /etc/default/grub
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="console=tty0,115200n8 crashkernel=auto no_timer_check net.ifnames=0"
GRUB_DISABLE_RECOVERY="true"


env|grep DIB
DIB_LOCAL_IMAGE=/home/stack/sasha/rhel-guest-image-7.6-215.x86_64.qcow2
DIB_YUM_REPO_CONF=/etc/yum.repos.d/rhos-release-13.repo /etc/yum.repos.d/rhos-release-rhel-7.6.repo


openstack overcloud image build --image-name overcloud-hardened-full --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel7.yaml --verbose 


Ends with error:
2018-10-25 17:29:52.808 | INFO diskimage_builder.block_device.blockdevice [-] Wrote final block device config to [/tmp/dib_build.gISylkT1/states/block-device/config.json]
2018-10-25 17:29:54.185 | INFO diskimage_builder.block_device.blockdevice [-] create() called
2018-10-25 17:29:54.190 | INFO diskimage_builder.block_device.level0.localloop [-] Create image file [/tmp/dib_image.e48mtbV2/image0.raw]
2018-10-25 17:29:54.191 | INFO diskimage_builder.block_device.level0.localloop [-] loopdev attach
2018-10-25 17:29:54.191 | INFO diskimage_builder.block_device.utils [-] Calling [sudo losetup --show -f /tmp/dib_image.e48mtbV2/image0.raw]
2018-10-25 17:29:54.239 | INFO diskimage_builder.block_device.level0.localloop [-] New block device [/dev/loop2]
2018-10-25 17:29:54.240 | INFO diskimage_builder.block_device.level1.partitioning [-] Creating partition on [image0] [/tmp/dib_image.e48mtbV2/image0.raw]
2018-10-25 17:29:54.241 | INFO diskimage_builder.block_device.level1.mbr [-] Create MBR disk partitioning object
2018-10-25 17:29:54.241 | INFO diskimage_builder.block_device.level1.mbr [-] Partition absolute [2048] relative [0] length [44921875] absolute end [44924928]
2018-10-25 17:29:54.242 | INFO diskimage_builder.block_device.level1.mbr [-] Write partition entry blockno [0] entry [0] start [2048] length [44921875]
2018-10-25 17:29:54.242 | INFO diskimage_builder.block_device.utils [-] Calling [sudo sync]
2018-10-25 17:29:54.295 | INFO diskimage_builder.block_device.utils [-] Calling [sudo kpartx -avs /dev/loop2]
2018-10-25 17:29:54.387 | INFO diskimage_builder.block_device.utils [-] Calling [sudo pvcreate /dev/mapper/loop2p1 --force]
2018-10-25 17:29:54.502 | INFO diskimage_builder.block_device.utils [-] Calling [sudo vgcreate vg /dev/mapper/loop2p1 --force]
2018-10-25 17:29:54.584 | ERROR diskimage_builder.block_device.blockdevice [-] Create failed; rollback initiated
2018-10-25 17:29:54.584 | Traceback (most recent call last):
2018-10-25 17:29:54.584 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/blockdevice.py", line 406, in cmd_create
2018-10-25 17:29:54.584 |     node.create()
2018-10-25 17:29:54.585 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/level1/lvm.py", line 278, in create
2018-10-25 17:29:54.585 |     vgs._create()
2018-10-25 17:29:54.585 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/level1/lvm.py", line 147, in _create
2018-10-25 17:29:54.585 |     exec_sudo(cmd)
2018-10-25 17:29:54.585 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py", line 143, in exec_sudo
2018-10-25 17:29:54.585 |     raise e
2018-10-25 17:29:54.585 | BlockDeviceSetupException: exec_sudo failed
2018-10-25 17:29:54.585 | INFO diskimage_builder.block_device.level0.localloop [-] loopdev detach
2018-10-25 17:29:54.585 | INFO diskimage_builder.block_device.utils [-] Calling [sudo losetup -d /dev/loop2]
2018-10-25 17:29:54.642 | INFO diskimage_builder.block_device.level0.localloop [-] Remove image file [/tmp/dib_image.e48mtbV2/image0.raw]
2018-10-25 17:29:54.648 | ERROR diskimage_builder.block_device.blockdevice [-] Rollback complete, exiting
2018-10-25 17:29:54.649 | Traceback (most recent call last):
2018-10-25 17:29:54.649 |   File "/bin/dib-block-device", line 10, in <module>
2018-10-25 17:29:54.649 |     sys.exit(main())
2018-10-25 17:29:54.649 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 120, in main
2018-10-25 17:29:54.649 |     return bdc.main()
2018-10-25 17:29:54.649 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 115, in main
2018-10-25 17:29:54.649 |     self.args.func()
2018-10-25 17:29:54.649 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 36, in cmd_create
2018-10-25 17:29:54.649 |     self.bd.cmd_create()
2018-10-25 17:29:54.649 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/blockdevice.py", line 406, in cmd_create
2018-10-25 17:29:54.650 |     node.create()
2018-10-25 17:29:54.650 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/level1/lvm.py", line 278, in create
2018-10-25 17:29:54.650 |     vgs._create()
2018-10-25 17:29:54.650 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/level1/lvm.py", line 147, in _create
2018-10-25 17:29:54.650 |     exec_sudo(cmd)
2018-10-25 17:29:54.650 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py", line 143, in exec_sudo
2018-10-25 17:29:54.650 |     raise e
2018-10-25 17:29:54.650 | diskimage_builder.block_device.exception.BlockDeviceSetupException: exec_sudo failed
2018-10-25 17:29:55.917 | INFO diskimage_builder.block_device.blockdevice [-] State already cleaned - no way to do anything here
2018-10-25 17:29:55.952 | *** /tmp/dib_build.gISylkT1/mnt is not a directory
2018-10-25 17:29:59.824 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:29:59.824 |         (In some cases useful info about processes that use
2018-10-25 17:29:59.824 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:00.885 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:00.885 |         (In some cases useful info about processes that use
2018-10-25 17:30:00.885 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:01.942 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:01.942 |         (In some cases useful info about processes that use
2018-10-25 17:30:01.942 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:03.001 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:03.001 |         (In some cases useful info about processes that use
2018-10-25 17:30:03.001 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:04.066 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:04.066 |         (In some cases useful info about processes that use
2018-10-25 17:30:04.066 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:05.124 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:05.124 |         (In some cases useful info about processes that use
2018-10-25 17:30:05.124 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:06.180 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:06.181 |         (In some cases useful info about processes that use
2018-10-25 17:30:06.181 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:07.238 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:07.238 |         (In some cases useful info about processes that use
2018-10-25 17:30:07.238 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:08.299 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:08.299 |         (In some cases useful info about processes that use
2018-10-25 17:30:08.299 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:09.356 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:09.356 |         (In some cases useful info about processes that use
2018-10-25 17:30:09.356 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:10.415 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:10.415 |         (In some cases useful info about processes that use
2018-10-25 17:30:10.415 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:11.472 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:11.472 |         (In some cases useful info about processes that use
2018-10-25 17:30:11.472 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:12.528 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:12.528 |         (In some cases useful info about processes that use
2018-10-25 17:30:12.528 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:13.584 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:13.584 |         (In some cases useful info about processes that use
2018-10-25 17:30:13.585 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:14.642 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:14.642 |         (In some cases useful info about processes that use
2018-10-25 17:30:14.642 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:15.697 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:15.697 |         (In some cases useful info about processes that use
2018-10-25 17:30:15.697 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:16.753 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:16.753 |         (In some cases useful info about processes that use
2018-10-25 17:30:16.753 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:17.810 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:17.810 |         (In some cases useful info about processes that use
2018-10-25 17:30:17.810 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:18.867 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:18.867 |         (In some cases useful info about processes that use
2018-10-25 17:30:18.867 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:19.923 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:19.923 |         (In some cases useful info about processes that use
2018-10-25 17:30:19.923 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:20.979 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:20.979 |         (In some cases useful info about processes that use
2018-10-25 17:30:20.979 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:22.035 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:22.036 |         (In some cases useful info about processes that use
2018-10-25 17:30:22.036 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:23.100 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:23.100 |         (In some cases useful info about processes that use
2018-10-25 17:30:23.100 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:24.158 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:24.158 |         (In some cases useful info about processes that use
2018-10-25 17:30:24.158 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:25.214 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:25.214 |         (In some cases useful info about processes that use
2018-10-25 17:30:25.214 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:26.268 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:26.268 |         (In some cases useful info about processes that use
2018-10-25 17:30:26.268 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:27.326 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:27.326 |         (In some cases useful info about processes that use
2018-10-25 17:30:27.326 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:28.380 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:28.380 |         (In some cases useful info about processes that use
2018-10-25 17:30:28.380 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:29.410 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:29.410 |         (In some cases useful info about processes that use
2018-10-25 17:30:29.410 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:30.437 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:30.437 |         (In some cases useful info about processes that use
2018-10-25 17:30:30.437 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:31.490 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:31.490 |         (In some cases useful info about processes that use
2018-10-25 17:30:31.490 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:32.545 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:32.545 |         (In some cases useful info about processes that use
2018-10-25 17:30:32.545 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:33.601 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:33.601 |         (In some cases useful info about processes that use
2018-10-25 17:30:33.601 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:34.657 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:34.657 |         (In some cases useful info about processes that use
2018-10-25 17:30:34.657 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:35.716 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:35.716 |         (In some cases useful info about processes that use
2018-10-25 17:30:35.716 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:36.776 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:36.777 |         (In some cases useful info about processes that use
2018-10-25 17:30:36.777 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:37.834 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:37.834 |         (In some cases useful info about processes that use
2018-10-25 17:30:37.834 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:38.893 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:38.893 |         (In some cases useful info about processes that use
2018-10-25 17:30:38.893 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:39.950 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:39.950 |         (In some cases useful info about processes that use
2018-10-25 17:30:39.950 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:41.008 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:41.008 |         (In some cases useful info about processes that use
2018-10-25 17:30:41.008 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:42.065 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:42.066 |         (In some cases useful info about processes that use
2018-10-25 17:30:42.066 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:43.122 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:43.122 |         (In some cases useful info about processes that use
2018-10-25 17:30:43.122 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:44.181 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:44.181 |         (In some cases useful info about processes that use
2018-10-25 17:30:44.181 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:45.242 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:45.242 |         (In some cases useful info about processes that use
2018-10-25 17:30:45.242 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:46.300 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:46.300 |         (In some cases useful info about processes that use
2018-10-25 17:30:46.300 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:47.357 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:47.358 |         (In some cases useful info about processes that use
2018-10-25 17:30:47.358 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:48.417 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:48.417 |         (In some cases useful info about processes that use
2018-10-25 17:30:48.417 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:49.476 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:49.476 |         (In some cases useful info about processes that use
2018-10-25 17:30:49.476 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:50.533 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:50.533 |         (In some cases useful info about processes that use
2018-10-25 17:30:50.533 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:51.589 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:51.589 |         (In some cases useful info about processes that use
2018-10-25 17:30:51.590 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:52.643 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:52.644 |         (In some cases useful info about processes that use
2018-10-25 17:30:52.644 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:53.698 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:53.698 |         (In some cases useful info about processes that use
2018-10-25 17:30:53.698 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:54.753 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:54.753 |         (In some cases useful info about processes that use
2018-10-25 17:30:54.753 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:55.809 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:55.809 |         (In some cases useful info about processes that use
2018-10-25 17:30:55.809 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:56.864 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:56.864 |         (In some cases useful info about processes that use
2018-10-25 17:30:56.864 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:57.920 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:57.920 |         (In some cases useful info about processes that use
2018-10-25 17:30:57.920 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:30:58.979 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:30:58.979 |         (In some cases useful info about processes that use
2018-10-25 17:30:58.979 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:00.035 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:00.035 |         (In some cases useful info about processes that use
2018-10-25 17:31:00.035 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:01.092 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:01.092 |         (In some cases useful info about processes that use
2018-10-25 17:31:01.092 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:02.149 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:02.149 |         (In some cases useful info about processes that use
2018-10-25 17:31:02.149 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:03.209 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:03.209 |         (In some cases useful info about processes that use
2018-10-25 17:31:03.209 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:04.267 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:04.267 |         (In some cases useful info about processes that use
2018-10-25 17:31:04.267 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:05.330 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:05.330 |         (In some cases useful info about processes that use
2018-10-25 17:31:05.330 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:06.388 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:06.388 |         (In some cases useful info about processes that use
2018-10-25 17:31:06.388 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:07.444 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:07.444 |         (In some cases useful info about processes that use
2018-10-25 17:31:07.444 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:08.499 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:08.499 |         (In some cases useful info about processes that use
2018-10-25 17:31:08.499 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:09.558 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:09.558 |         (In some cases useful info about processes that use
2018-10-25 17:31:09.558 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:10.620 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:10.620 |         (In some cases useful info about processes that use
2018-10-25 17:31:10.620 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:11.676 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:11.676 |         (In some cases useful info about processes that use
2018-10-25 17:31:11.676 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:12.733 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:12.733 |         (In some cases useful info about processes that use
2018-10-25 17:31:12.733 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:13.794 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:13.794 |         (In some cases useful info about processes that use
2018-10-25 17:31:13.794 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:14.850 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:14.850 |         (In some cases useful info about processes that use
2018-10-25 17:31:14.850 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:15.896 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:15.896 |         (In some cases useful info about processes that use
2018-10-25 17:31:15.896 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:16.954 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:16.954 |         (In some cases useful info about processes that use
2018-10-25 17:31:16.954 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:18.010 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:18.010 |         (In some cases useful info about processes that use
2018-10-25 17:31:18.011 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:19.069 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:19.069 |         (In some cases useful info about processes that use
2018-10-25 17:31:19.069 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:20.124 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:20.124 |         (In some cases useful info about processes that use
2018-10-25 17:31:20.124 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:21.184 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:21.184 |         (In some cases useful info about processes that use
2018-10-25 17:31:21.184 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:22.239 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:22.240 |         (In some cases useful info about processes that use
2018-10-25 17:31:22.240 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:23.295 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:23.295 |         (In some cases useful info about processes that use
2018-10-25 17:31:23.295 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:24.353 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:24.353 |         (In some cases useful info about processes that use
2018-10-25 17:31:24.353 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:25.412 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:25.412 |         (In some cases useful info about processes that use
2018-10-25 17:31:25.412 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:26.477 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:26.477 |         (In some cases useful info about processes that use
2018-10-25 17:31:26.477 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:27.536 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:27.536 |         (In some cases useful info about processes that use
2018-10-25 17:31:27.536 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:28.590 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:28.591 |         (In some cases useful info about processes that use
2018-10-25 17:31:28.591 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:29.646 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:29.646 |         (In some cases useful info about processes that use
2018-10-25 17:31:29.646 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:30.702 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:30.702 |         (In some cases useful info about processes that use
2018-10-25 17:31:30.702 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:31.757 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:31.758 |         (In some cases useful info about processes that use
2018-10-25 17:31:31.758 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:32.788 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:32.788 |         (In some cases useful info about processes that use
2018-10-25 17:31:32.788 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:33.843 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:33.844 |         (In some cases useful info about processes that use
2018-10-25 17:31:33.844 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:34.900 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:34.900 |         (In some cases useful info about processes that use
2018-10-25 17:31:34.900 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:35.962 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:35.962 |         (In some cases useful info about processes that use
2018-10-25 17:31:35.962 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:37.021 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:37.021 |         (In some cases useful info about processes that use
2018-10-25 17:31:37.021 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:38.079 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:38.079 |         (In some cases useful info about processes that use
2018-10-25 17:31:38.079 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:39.136 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:39.136 |         (In some cases useful info about processes that use
2018-10-25 17:31:39.136 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:40.191 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:40.191 |         (In some cases useful info about processes that use
2018-10-25 17:31:40.191 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:41.249 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:41.249 |         (In some cases useful info about processes that use
2018-10-25 17:31:41.249 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:42.307 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:42.307 |         (In some cases useful info about processes that use
2018-10-25 17:31:42.307 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:43.365 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:43.365 |         (In some cases useful info about processes that use
2018-10-25 17:31:43.365 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:44.423 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:44.423 |         (In some cases useful info about processes that use
2018-10-25 17:31:44.423 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:45.479 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:45.479 |         (In some cases useful info about processes that use
2018-10-25 17:31:45.479 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:46.539 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:46.539 |         (In some cases useful info about processes that use
2018-10-25 17:31:46.539 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:47.598 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:47.598 |         (In some cases useful info about processes that use
2018-10-25 17:31:47.598 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:48.655 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:48.655 |         (In some cases useful info about processes that use
2018-10-25 17:31:48.655 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:49.712 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:49.713 |         (In some cases useful info about processes that use
2018-10-25 17:31:49.713 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:50.770 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:50.770 |         (In some cases useful info about processes that use
2018-10-25 17:31:50.770 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:51.827 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:51.827 |         (In some cases useful info about processes that use
2018-10-25 17:31:51.827 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:52.883 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:52.883 |         (In some cases useful info about processes that use
2018-10-25 17:31:52.883 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:53.940 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:53.940 |         (In some cases useful info about processes that use
2018-10-25 17:31:53.940 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:54.995 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:54.995 |         (In some cases useful info about processes that use
2018-10-25 17:31:54.995 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:56.051 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:56.051 |         (In some cases useful info about processes that use
2018-10-25 17:31:56.051 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:57.107 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:57.107 |         (In some cases useful info about processes that use
2018-10-25 17:31:57.107 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:58.165 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:58.165 |         (In some cases useful info about processes that use
2018-10-25 17:31:58.166 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:59.222 | umount: /tmp/dib_image.e48mtbV2: target is busy.
2018-10-25 17:31:59.222 |         (In some cases useful info about processes that use
2018-10-25 17:31:59.222 |          the device is found by lsof(8) or fuser(1))
2018-10-25 17:31:59.773 | ERROR: failed to umount the /tmp/dib_image.e48mtbV2 tmpfs mount point
Command '['disk-image-create', '-a', 'amd64', '-o', './overcloud-hardened-full', '-t', 'qcow2', '-p', 'ntp,python-psutil,python-debtcollector,plotnetcfg,sos,device-mapper-multipath,python-heat-agent-puppet,python-heat-agent-hiera,python-heat-agent-apply-config,python-heat-agent-ansible,python-heat-agent-docker-cmd,python-heat-agent-json-file,screen,os-net-config,jq', '--min-tmpfs=7', 'rhel7', 'dhcp-all-interfaces', 'openvswitch', 'overcloud-agent', 'overcloud-full', 'overcloud-controller', 'overcloud-compute', 'overcloud-ceph-storage', 'puppet-modules', 'stable-interface-names', 'bootloader', 'element-manifest', 'dynamic-login', 'iptables', 'enable-packages-install', 'pip-and-virtualenv-override', 'dracut-regenerate', 'remove-machine-id', 'remove-resolvconf', 'modprobe-blacklist', 'overcloud-secure']' returned non-zero exit status 1
END return value: 1

Comment 1 Alexander Chuzhoy 2018-11-19 17:40:56 UTC

After rebooting the undercloud node - successfully built the image.

Comment 2 Luigi Toscano 2018-11-22 18:22:00 UTC

For the record, I hit the same issue when using sahara-image-create, which is a wrapper around diskimage-builder, on RHEL 7.6:

2018-11-22 17:41:50.840 | INFO diskimage_builder.block_device.utils [-] Calling [sudo umount /tmp/dib_build.d2AkFELg/mnt/]
2018-11-22 17:41:50.858 | DEBUG diskimage_builder.block_device.utils [-] exec_sudo: umount: /tmp/dib_build.d2AkFELg/mnt: target is busy. exec_sudo /usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py:135
2018-11-22 17:41:50.858 | DEBUG diskimage_builder.block_device.utils [-] exec_sudo:         (In some cases useful info about processes that use exec_sudo /usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py:135
2018-11-22 17:41:50.858 | DEBUG diskimage_builder.block_device.utils [-] exec_sudo:          the device is found by lsof(8) or fuser(1)) exec_sudo /usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py:135
2018-11-22 17:41:50.858 | Traceback (most recent call last):
2018-11-22 17:41:50.858 |   File "/bin/dib-block-device", line 10, in <module>
2018-11-22 17:41:50.858 |     sys.exit(main())
2018-11-22 17:41:50.858 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 120, in main
2018-11-22 17:41:50.859 |     return bdc.main()
2018-11-22 17:41:50.859 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 115, in main
2018-11-22 17:41:50.859 |     self.args.func()
2018-11-22 17:41:50.859 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/cmd.py", line 39, in cmd_umount
2018-11-22 17:41:50.859 |     self.bd.cmd_umount()
2018-11-22 17:41:50.859 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/blockdevice.py", line 441, in cmd_umount
2018-11-22 17:41:50.859 |     node.umount()
2018-11-22 17:41:50.859 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/level3/mount.py", line 112, in umount
2018-11-22 17:41:50.859 |     exec_sudo(["umount", self.state['mount'][self.mount_point]['path']])
2018-11-22 17:41:50.860 |   File "/usr/lib/python2.7/site-packages/diskimage_builder/block_device/utils.py", line 143, in exec_sudo
2018-11-22 17:41:50.870 |     raise e
2018-11-22 17:41:50.870 | diskimage_builder.block_device.exception.BlockDeviceSetupException: exec_sudo failed


Packages:
sahara-image-elements-9.0.1-0.20180830223845.4b48079.el7ost.noarch
diskimage-builder-2.17.1-0.20181011095309.bbbade0.el7ost.noarch
dib-utils-0.0.11-1.el7ost.noarch

A restart of the node did not help.

Comment 3 Luigi Toscano 2018-11-22 18:23:34 UTC

I apologize: my case it's not OSP13, so it may not be relevant here. Still the error looks the same, so it may be something that changed in RHEL.

Comment 4 Luigi Toscano 2018-11-23 11:21:23 UTC

Update: I could reproduce the same issue on OSP13, and a reboot did not help.

sahara-image-elements-8.0.1-2.el7ost.noarch
diskimage-builder-2.16.0-2.el7ost.noarch
dib-utils-0.0.11-1.el7ost.noarch

Comment 5 Yolanda Robla 2018-11-23 11:29:43 UTC

It will fail on every image build based on this RHEL 7.6 image, as there is currently a syntax error on their grub defaults: https://bugzilla.redhat.com/show_bug.cgi?id=1646976

> > GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto" console=ttyS0,115200n8
> > no_timer_check net.ifnames=0
> > 
> > As you can see, this syntax is wrong, as the quotes are not covering all the
> > parameters. The exact syntax needs to be:
> > 
> > GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto console=ttyS0,115200n8
> > no_timer_check net.ifnames=0"

Comment 6 Luigi Toscano 2018-11-23 13:40:05 UTC

I confirm that if I fix the content of /etc/default/grub on the image, sahara-image-create (aka diskimage-builder) works (on OSP13 and later).

Should this bug be reassigned to the other product and closed as duplicate of rhbz#1646976 ?

Comment 7 Bob Fournier 2018-11-26 18:40:10 UTC

Closing this out. https://bugzilla.redhat.com/show_bug.cgi?id=1643273 is the same issue and has also been closed.  The RHEL 7.6 bug https://bugzilla.redhat.com/show_bug.cgi?id=1646976 is the issue here.

*** This bug has been marked as a duplicate of bug 1646976 ***