Hide Forgot
Description of problem: ironic.drivers.modules.iscsi_deploy Deploy failed for instance Error: Unexpected error while running command. Version-Release number of selected component (if applicable): [stack@director1 log]$ rpm -qa | grep ironic python-ironicclient-0.5.1-12.el7ost.noarch openstack-ironic-discoverd-1.1.0-8.el7ost.noarch openstack-ironic-common-2015.1.2-2.el7ost.noarch openstack-ironic-api-2015.1.2-2.el7ost.noarch openstack-ironic-conductor-2015.1.2-2.el7ost.noarch python-ironic-discoverd-1.1.0-8.el7ost.noarch How reproducible: every Steps to Reproduce: 1. 2. 3. Actual results: Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.541 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Starting to partition the disk device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-1 0.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:244 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.542 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Add config drive partition (1 MB) to device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn. 2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:271 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.542 1242 DEBUG ironic.drivers.modules.deploy_utils [-] Add root partition (3814400 MB) to device: /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.20 08-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 make_partitions /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:279 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.543 1242 DEBUG ironic.common.disk_partitioner [-] Committing partitions to disk. commit /usr/lib/python2.7/site-packages/ironic/common/disk_partiti oner.py:138 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.543 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev /disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary 1 2 mkpart primary 2 3814402 set 2 boot on execute /usr/lib/ python2.7/site-packages/oslo_concurrency/processutils.py:223 Apr 4 07:09:31 director1 iscsid: Could not set session37 priority. READ/WRITE throughout and latency could be affected. Apr 4 07:09:31 director1 iscsid: Connection37:0 to [target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260] through [iface: default] is operational now Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.689 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3 .3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary 1 2 mkpart primary 2 3814402 set 2 boot on" returned: 1 in 0.146s execute /usr/li b/python2.7/site-packages/oslo_concurrency/processutils.py:254 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.690 1242 DEBUG oslo_concurrency.processutils [-] u'sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3. 134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary 1 2 mkpart primary 2 3814402 set 2 boot on' failed. Not Retrying. execute /usr/lib/p ython2.7/site-packages/oslo_concurrency/processutils.py:291 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.690 1242 ERROR ironic.drivers.modules.deploy_utils [-] Deploy to address 10.3.3.134 failed. Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary 1 2 mkpart primary 2 3814402 set 2 boot on Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] StdOut: u'' Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.691 1242 ERROR ironic.drivers.modules.deploy_utils [-] StdErr: u'Warning: The resulting partition is not properly aligned for best performance.\nError: The location 3814402 is outside of the device /dev/sda.\n' Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.692 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:223 Apr 4 07:09:31 director1 kernel: sd 38:0:0:1: alua: Detached Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.839 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout" returned: 0 in 0.147s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:254 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.840 1242 DEBUG ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c --logout" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.840 1242 DEBUG ironic.common.utils [-] Command stdout is: "Logging out of session [sid: 37, target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260] Apr 4 07:09:31 director1 ironic-conductor: Logout of [sid: 37, target: iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c, portal: 10.3.3.134,3260] successful. Apr 4 07:09:31 director1 ironic-conductor: " execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.841 1242 DEBUG ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.841 1242 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:223 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.968 1242 DEBUG oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete" returned: 0 in 0.126s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:254 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.968 1242 DEBUG ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 10.3.3.134:3260 -T iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c -o delete" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 DEBUG ironic.common.utils [-] Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 DEBUG ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85 Apr 4 07:09:31 director1 ironic-conductor: 2016-04-04 07:09:31.969 1242 ERROR ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance 7953f2b9-decd-4362-8d73-f55d342c8513. Error: Unexpected error while running command. Apr 4 07:09:31 director1 ironic-conductor: Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-10.3.3.134:3260-iscsi-iqn.2008-10.org.openstack:f24326b9-f902-4ed8-8bfe-6b62808e353c-lun-1 -- unit MiB mklabel msdos mkpart primary 1 2 mkpart primary 2 3814402 set 2 boot on Apr 4 07:09:31 director1 ironic-conductor: Exit code: 1 Expected results: Additional info:
Hi! What is your real disk size and what is in the Ironic properties (local_gb) field?
introspection (when it was working) was reporting 277GB and that's what 'ironic node-show' was reporting, too. When we get introspection working again (see all the other bugs), I can set this to 276, manually. I know I read about this size mismatch problem somewhere - maybe another BZ - is it documented or, better yet, taken care of automatically by ironic-inspector?
Introspection usually removes 1 GB from the real size to account for partitioning. But what's the real (physical) size in GB of the hard drive?
I don't have a v7 env to test this against anymore.
Got it, thanks for the update. Please reopen if you see it again.
Ran into this error in an OSP 13 deployment. The flavor size for the overcloud node being provisioned had a disk size that was > the actual size of the physical disk. See this KB: https://access.redhat.com/solutions/4486531