Bug 1471382 - IscsiDeploy can leave orphaned iscsi sessions [NEEDINFO]
IscsiDeploy can leave orphaned iscsi sessions
Status: CLOSED CURRENTRELEASE
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: RHOS Maint
mlammon
: Triaged
Depends On:
Blocks: 1475433
  Show dependency treegraph
 
Reported: 2017-07-15 08:31 EDT by Robin Cernin
Modified: 2017-10-18 13:52 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1475433 (view as bug list)
Environment:
Last Closed: 2017-08-07 11:31:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
bfournie: needinfo? (rcernin)


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1632649 None None None 2017-07-15 08:31 EDT
OpenStack gerrit 388092 None None None 2017-07-15 08:32 EDT

  None (edit)
Description Robin Cernin 2017-07-15 08:31:03 EDT
Description of problem:

Discovered while debugging iSCSI deploy failures.

Deploy fails with:

"iSCSI connection was not seen by the file system after attempting to verify 3 times."

2017-07-14 17:47:18.271 1494 ERROR ironic.drivers.modules.agent_base_vendor   File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 364, in _fail_deploy
2017-07-14 17:47:18.271 1494 ERROR ironic.drivers.modules.agent_base_vendor     raise exception.InstanceDeployFailure(msg)
2017-07-14 17:47:18.271 1494 ERROR ironic.drivers.modules.agent_base_vendor InstanceDeployFailure: Deploy failed for instance f1c9feed-06f2-4f6b-bb40-683df39e90dd. Error: iSCSI connection was not seen by the file system after attempting to verify 3 times.

Running iscsiadm shows many iSCSI sessions

$ iscsiadm -m session
tcp: [10] xxxx:3260,1 iqn.2008-10.org.openstack:5255dc8a-0638-403b-83e7-5a7726d93873 (non-flash)
tcp: [100] xxxx:3260,1 iqn.2008-10.org.openstack:db9081a8-0aac-43d5-b792-03b964b20cb0 (non-flash)
tcp: [101] xxxx:3260,1 iqn.2008-10.org.openstack:bbe97782-ff4d-4f21-a533-1b36dfb973db (non-flash)
tcp: [102] xxxx:3260,1 iqn.2008-10.org.openstack:3e174b2e-855b-4e7d-aac7-df474664074b (non-flash)
...

These are presumably from attempts to deploy the nodes, but Ironic has not closed/cleaned up those sessions on deploy failure.

Version-Release number of selected component (if applicable):

openstack-ironic-api-4.2.5-3.el7ost.noarch                  Thu Jul  6 14:50:52 2017
openstack-ironic-common-4.2.5-3.el7ost.noarch               Thu Jul  6 14:42:24 2017
openstack-ironic-conductor-4.2.5-3.el7ost.noarch            Thu Jul  6 14:42:24 2017
openstack-ironic-inspector-2.2.6-1.el7ost.noarch            Thu Jul  6 14:50:47 2017
python-ironic-inspector-client-1.2.0-6.el7ost.noarch        Thu Jul  6 14:32:21 2017
python-ironicclient-0.8.1-1.el7ost.noarch                   Thu Jul  6 14:32:20 2017
Comment 2 Bob Fournier 2017-08-04 12:27:43 EDT
Hi Robin - it looks like there is a fix for this upstream as you've indicated in the external trackers. https://review.openstack.org/#/c/388092/.

Are you requesting a backport?
Comment 3 Bob Fournier 2017-08-07 11:31:38 EDT
Closing as fixed in current release.

Note You need to log in before you can comment on or make changes to this bug.