Bug 1324241

Summary:	Deploy failed when running parted: The location 3814402 is outside of the device /dev/sda
Product:	Red Hat OpenStack	Reporter:	Dan Yocum <dyocum>
Component:	openstack-ironic	Assignee:	Lucas Alvares Gomes <lmartins>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Raviv Bar-Tal <rbartal>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	7.0 (Kilo)	CC:	cfields, dhill, dtantsur, dyocum, mburns, rhel-osp-director-maint, srevivo
Target Milestone:	---	Keywords:	ZStream
Target Release:	8.0 (Liberty)
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-05-18 08:34:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Dan Yocum 2016-04-05 23:05:05 UTC

Description of problem:

ironic.drivers.modules.iscsi_deploy Deploy failed for instance Error: Unexpected error while running command.

Version-Release number of selected component (if applicable):

[stack@director1 log]$ rpm -qa | grep ironic
python-ironicclient-0.5.1-12.el7ost.noarch
openstack-ironic-discoverd-1.1.0-8.el7ost.noarch
openstack-ironic-common-2015.1.2-2.el7ost.noarch
openstack-ironic-api-2015.1.2-2.el7ost.noarch
openstack-ironic-conductor-2015.1.2-2.el7ost.noarch
python-ironic-discoverd-1.1.0-8.el7ost.noarch

How reproducible:

every

Steps to Reproduce:
1. 
2.
3.

Actual results:

Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.541 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Starting to partition the disk device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-1
0.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:244
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.542 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Add config drive partition (1 MB) to device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.
2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:271
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.542 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Add root partition (3814400 MB) to device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.20
08-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:279
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.543 1242 DEBUG ironic.common.disk_partitioner [-] Committing partitions to disk. commit /usr/lib/python2.7/site-packages/ironic/common/disk_partiti
oner.py:138
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.543 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev
/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary  1 2 mkpart primary  2 3814402 set 2 boot on execute /usr/lib/
python2.7/site-packages/oslo_concurrency/processutils.py:223
Apr  4 07:09:31 director1 iscsid: Could not set session37 priority. READ/WRITE throughout and latency could be affected.
Apr  4 07:09:31 director1 iscsid: Connection37:0 to [target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260] through [iface: default] is operational now
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.689 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3
.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary  1 2 mkpart primary  2 3814402 set 2 boot on" returned: 1 in 0.146s execute /usr/li
b/python2.7/site-packages/oslo_concurrency/processutils.py:254
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.690 1242 DEBUG oslo_concurrency.processutils [-] u'sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3.
134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary  1 2 mkpart primary  2 3814402 set 2 boot on' failed. Not Retrying. execute /usr/lib/p
ython2.7/site-packages/oslo_concurrency/processutils.py:291
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.690 1242 ERROR ironic.drivers.modules.deploy_utils [-] Deploy to address 10.3.3.134 failed.
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary  1 2 mkpart primary  2 3814402 set 2 boot on
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] StdOut: u''
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] StdErr: u'Warning: The resulting partition is not properly aligned for best performance.\nError: The location 3814402 is outside of the device /dev/sda.\n'
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.692 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:223
Apr  4 07:09:31 director1 kernel: sd 38:0:0:1: alua: Detached
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.839 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout" returned: 0 in 0.147s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:254
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.840 1242 DEBUG ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.840 1242 DEBUG ironic.common.utils [-] Command stdout is: "Logging out of session [sid: 37, target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260]
Apr  4 07:09:31 director1 ironic-conductor: Logout of [sid: 37, target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260] successful.
Apr  4 07:09:31 director1 ironic-conductor: " execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.841 1242 DEBUG ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.841 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:223
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.968 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete" returned: 0 in 0.126s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:254
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.968 1242 DEBUG ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 DEBUG ironic.common.utils [-] Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 DEBUG ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85
Apr  4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 ERROR ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance 7953f2b9-decd-4362-8d73-f55d342c8513. Error: Unexpected error while running command.
Apr  4 07:09:31 director1 ironic-conductor: Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary  1 2 mkpart primary  2 3814402 set 2 boot on
Apr  4 07:09:31 director1 ironic-conductor: Exit code: 1


Expected results:


Additional info:

Comment 3 Dmitry Tantsur 2016-04-18 08:20:09 UTC

Hi!

What is your real disk size and what is in the Ironic properties (local_gb) field?

Comment 4 Dan Yocum 2016-04-18 19:54:10 UTC

introspection (when it was working) was reporting 277GB and that's what 'ironic node-show' was reporting, too.  

When we get introspection working again (see all the other bugs), I can set this to 276, manually.  I know I read about this size mismatch problem somewhere - maybe another BZ - is it documented or, better yet, taken care of automatically by ironic-inspector?

Comment 5 Dmitry Tantsur 2016-05-09 12:37:44 UTC

Introspection usually removes 1 GB from the real size to account for partitioning. But what's the real (physical) size in GB of the hard drive?

Comment 6 Dan Yocum 2016-05-17 19:44:38 UTC

I don't have a v7 env to test this against anymore.

Comment 7 Dmitry Tantsur 2016-05-18 08:34:40 UTC

Got it, thanks for the update. Please reopen if you see it again.

Comment 8 Chris Fields 2019-10-09 13:12:55 UTC

Ran into this error in an OSP 13 deployment.  The flavor size for the overcloud node being provisioned had a disk size that was > the actual size of the physical disk.  See this KB: https://access.redhat.com/solutions/4486531