Bug 1568572

Summary: iscsiadm discovery command on storage appliance B causes logout from storage appliance A
Product: Red Hat Enterprise Linux 6 Reporter: paulp <paulp>
Component: iscsi-initiator-utilsAssignee: Chris Leech <cleech>
Status: CLOSED WONTFIX QA Contact: Filip Suba <fsuba>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.9CC: akarlsso, coughlan, paulp, toneata, tumeya
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-03 12:26:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1414018, 1425546    
Attachments:
Description Flags
Network trace and iscsi configuration data none

Description paulp@netapp.com 2018-04-17 19:17:57 UTC 
Description of problem:
Redhat 6.9 client (also verified happens with 6.6) is lun-booted from network storage device A, so root filesystem is an iscsi device.

Issue discovery command "iscsiadm --mode discovery --type sendtargets --portal <ip address>" to new network storage device B causes Redhat to log out the connection to storage device A.  Then there is a race for the Redhat client to iscsi re-login into device A before some amount of time expires (2 seconds?) and if the login doesn't happen in that time, the root filesystem becomes read-only.

Verified with Wireshark trace the RHEL client chooses to log out of the connection with device A when discovery command is issued with IP address of device B, and then a login operation immediately begins to reestablish the connection with device A.

Since this causes a service disruption I'm looking for a way to prevent the logout from device A when discovering luns on device B.

Version-Release number of selected component (if applicable):
RHEL 6.6 and 6.9

How reproducible:
Readily

Steps to Reproduce:
1. Lun boot RHEL 6.9 using iSCSI lun on network storage device A
2. Issue iscsiadm sendtargets command to network storage device B 
3. Observe iscsi connection to device A is terminated
4. Observe in a high number of instance the iscsi connection is not reestablished fast enough to prevent read-only root filesystem

Actual results:
Root filesystem becomes read-only and the client has to be rebooted.

Expected results:
Iscsi discovery to a storage device does not affect existing iscsi connections.

Additional info:
Comment 2 paulp@netapp.com 2018-05-18 14:10:27 UTC 
Reproduced with RHEL 6.10
Comment 3 Chris Leech 2018-05-22 20:19:21 UTC 
Can you give me more details on the initiator configuration?

Is this with iscsi_tcp or with an iSCSI HBA?
If it's an HBA, which make/driver?
Are the two targets on the same network, or different networks?
Does it reproduce without booting from iSCSI?

Can you share a network trace that shows the logout?
How about the iSCSI configuration from /var/lib/iscsi?

My initial attempts to reproduce this failed, in that I did not see this behavior.
Comment 4 paulp@netapp.com 2018-05-31 15:23:31 UTC 
Created attachment 1446312 [details]
Network trace and iscsi configuration data
Comment 5 paulp@netapp.com 2018-05-31 15:26:07 UTC 
Answering Chris Leech questions:
* This is with iscsi_tcp
* The two targets are on different networks
* I don't have a way to reproduce without booting from iSCSI
* I have attached an attachment with network trace and files from /var/lib/iscsi