RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 658428 - [NetApp 6.1 bug] /etc/init.d/iscsi script does not start/stop as per state machine when root is on iSCSI LUN
Summary: [NetApp 6.1 bug] /etc/init.d/iscsi script does not start/stop as per state ma...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: iscsi-initiator-utils
Version: 6.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: 6.1
Assignee: Andy Grover
QA Contact: Storage QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-11-30 10:51 UTC by gowrav
Modified: 2015-09-28 02:05 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Starting or stopping the iSCSI service while accessing the root partition directly through an iSCSI disk could cause iSCSI to become unresponsive and incorrect status information to be reported. Attempting to stop the iSCSI service in this circumstance now warns that iSCSI cannot be shut down while Root is on an iSCSI disk, and all statuses are reported correctly.
Clone Of:
Environment:
Last Closed: 2011-05-19 14:14:53 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:0733 0 normal SHIPPED_LIVE iscsi-initiator-utils bug fix and enhancement update 2011-05-19 09:47:49 UTC

Description gowrav 2010-11-30 10:51:25 UTC 
Description of problem:

In RHEL 6 GA OS, the /etc/init.d/iscsi script does not start/stop/restart as expected. The OS root is installed on iSCSI multipathed LUN, and all the iSCSI session have been configured to start "on boot". Below are different scenarios.

1) After boot, all the iSCSI sessions are established and the all LUNs are visible. But when we check for iscsi status, it shows as 'stopped'. Whereas it should have started during boot time.
# iscsiadm -m session
tcp: [1] 10.72.201.36:3260,1000 iqn.1992-08.com.netapp:sn.101180506
tcp: [2] 10.72.201.37:3260,1000 iqn.1992-08.com.netapp:sn.101180456
tcp: [3] 192.168.200.37:3260,1002 iqn.1992-08.com.netapp:sn.101180456
tcp: [4] 192.168.200.36:3260,1002 iqn.1992-08.com.netapp:sn.101180506

# /etc/init.d/iscsi status
	iscsi is stopped	

2) When I try to "start" the iscsi scripts, it fails to start due to a check condition in the script. 

# /etc/init.d/iscsi start
Starting iscsi: iscsiadm: Could not login to [iface: default, target: iqn.1992-08.com.netapp:sn.101180456, portal: 10.72.201.37,3260].
iscsiadm: initiator reported error (15 - already exists)
iscsiadm: Could not login to [iface: default, target: iqn.1992-08.com.netapp:sn.101180456, portal: 192.168.200.37,3260].
iscsiadm: initiator reported error (15 - already exists)
iscsiadm: Could not login to [iface: default, target: iqn.1992-08.com.netapp:sn.101180506, portal: 10.72.201.36,3260].
iscsiadm: initiator reported error (15 - already exists)
iscsiadm: Could not login to [iface: default, target: iqn.1992-08.com.netapp:sn.101180506, portal: 192.168.200.36,3260].
iscsiadm: initiator reported error (15 - already exists)
                                                           [FAILED]

iscsi is not starting since the check condition fails in the following lines. Seems like PIPESTATUS[0] is not returning correct status.

 if [ ${PIPESTATUS[0]} -ne 0 -o ${PIPESTATUS[1]} -eq 0 ]; then
        failure $"Starting $prog"
        echo
        return 1
    fi

Ideally, "iscsi" should start at boot time when all session are established and especially root is on a  iSCSI LUN. If its not started on boot time, iscsi should atleast start when it's started manually. This scenario is observed to work fine in RHEL 5 series.

3) If I try to 'restart' the "/etc/init.d/iscsi" script (after trying step-2), the host stops responding and i'm unable to execute any command thereafter. Here my root is on a iSCSI LUN.

# /etc/init.d/iscsi restart
	
Stopping iscsi:
	/etc/init.d/iscsi: line 80: rm: command not found

Message from syslogd@IBMblade-200-202 at Nov 29 17:21:01 ...
 kernel:journal commit I/O error

 # clear
-bash: /usr/bin/clear: No such file or directory


4) When I try to 'stop' the iscsi scripts when the root is on a iSCSI LUN, it's observed that iSCSI daemon stops by logging off all the iscsi sessions and host hangs. Ideally, the iSCSI daemon should NOT stop giving a message that "Root is on a iSCSI LUN and iSCSI cannot be stopped". But there is no such check condition in "/etc/init.d/iscsi" script. Such a check condition is present in RHEL 5 series.

Version-Release number of selected component (if applicable):

Kernel: 2.6.32-71.el6.x86_64
iSCSI: iscsi-initiator-utils-6.2.0.872-10.el6.x86_64
Device Mapper: device-mapper-1.02.53-8.el6.x86_64
Multipath: device-mapper-multipath-0.4.9-31.el6.x86_64

How reproducible: Always


Steps to Reproduce:
1. Install RHEL 6.0 OS with root on a iSCSI multipathed LUN.
2. Configure all the iSCSI session to start with "on boot".
3. Reboot the host and check if all the iSCSI sessions have been established (iscsiadm -m session).
4. Make sure that /etc/init.d/iscsid daemon is running.
4. Run /etc/init.d/iscsi script with various options -- status, start, restart, stop.
  
Actual results:

When root is installed on iSCSI multipathed LUN, after the host reboot we see that all the iscsi sessions are established, but the iscsi status is show as "iscsi is stopped". When we try to start it manually, iscsi fails to start. When we try to stop/restart, all the iscsi sessions are logged off and host hangs.

Expected results:

When the OS is installed on iSCSI multipathed LUN, the iscsi script should start automatically at boot time and the status should be "running". And, when the iscsi is attempted to stop, there should be a message displayed that "iSCSI cannot be stopped since root is on iSCSI LUN", and iscsi sessions should be logged off, i.e., daemon should not be stopped.

Additional info:
Comment 4 gowrav 2011-01-14 05:33:43 UTC 
Hi Mike, any updates on this bug?
Comment 5 Mike Christie 2011-01-14 07:42:43 UTC 
It's on my todo for 6.1. Kernel deadline is Fri, so after that I am going to on work on these type of bugs. All are known issues, so they should get fixed.
Comment 6 Mike Christie 2011-01-24 19:15:31 UTC 
Hey Gowrav,

Do you have your setup handy? If so could you run

iscsiadm -m node -T iqn.1992-08.com.netapp:sn.101180506

and attach the output?

And you did the iscsi root installation through anaconda, right? You did not do a local install then manually setup iscsi root boot, right?
Comment 7 gowrav 2011-01-25 06:48:53 UTC 
Hi Mike,

Yes, I installed the OS on iscsi root via anaconda.

Below is the (In reply to comment #6)
> Hey Gowrav,
> 
> Do you have your setup handy? If so could you run
> 
> iscsiadm -m node -T iqn.1992-08.com.netapp:sn.101180506
> 
> and attach the output?
> 
> And you did the iscsi root installation through anaconda, right? You did not do
> a local install then manually setup iscsi root boot, right?

Hi Mike,

Yes, I installed the OS on iscsi root via anaconda.

Below is the output of the cmd you asked for.

[root@IBMblade-200-202 ~]# iscsiadm -m node -T iqn.1992-08.com.netapp:sn.101180506
# BEGIN RECORD 2.0-87

node.name = iqn.1992-08.com.netapp:sn.101180506
node.tpgt = 1000
node.startup = automatic
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
node.discovery_address = 10.72.201.36
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.auth.authmethod = None
node.session.auth.username = <empty>
node.session.auth.password = <empty>
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 10.72.201.36
node.conn[0].port = 3260
node.conn[0].startup = automatic
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD
# BEGIN RECORD 2.0-872
node.name = iqn.1992-08.com.netapp:sn.101180506
node.tpgt = 1002
node.startup = automatic
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
node.discovery_address = 10.72.201.36
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.auth.authmethod = None
node.session.auth.username = <empty>
node.session.auth.password = <empty>
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 192.168.200.36
node.conn[0].port = 3260
node.conn[0].startup = automatic
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD
Comment 8 Mike Christie 2011-02-01 08:45:24 UTC 
This is fixed in iscsi-initiator-utils-6.2.0.872-14.el6. You can download it here
http://people.redhat.com/mchristi/iscsi/rhel6.1/iscsi-initiator-utils/
Comment 10 gowrav 2011-02-06 06:41:19 UTC 
I verified the "iscsi-initiator-utils-6.2.0.872-14.el6.x86_64" rpm patch. All the options (restart, stop, start)are working as expected for "status" option. 

1) If the root is on iSCSI multipth'd lun, the iscsi scripts provides correct output stating "Can not shutdown iSCSI. Root is on a iSCSI disk.". This works as expected.

2) When "/etc/init.d/iscsi status" option is used, instead of just giving one line status like -- "iscsid (pid ) is running..."; iscsi scripts is giving a verbose output on the state of all the iscsi sessions and interfaces. I believe such verbose o/p is not required.

[root@IBMblade-200-202 ~]# /etc/init.d/iscsi status
iSCSI Transport Class version 2.0-870
version 2.0-872
Target: iqn.1992-08.com.netapp:sn.101180506
        Current Portal: 10.72.201.36:3260,1000
        Persistent Portal: 10.72.201.36:3260,1000
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.domain:lnx200.202
                Iface IPaddress: 10.72.200.202
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 1
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 65536
                FirstBurstLength: 65536
                MaxBurstLength: 65536
                ImmediateData: Yes
                InitialR2T: No
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 3  State: running
                scsi3 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdb          State: running
                scsi3 Channel 00 Id 0 Lun: 1
                        Attached scsi disk sdc          State: running
                scsi3 Channel 00 Id 0 Lun: 2
                        Attached scsi disk sdd          State: running
                scsi3 Channel 00 Id 0 Lun: 3
                        Attached scsi disk sde          State: running
        Current Portal: 192.168.200.36:3260,1002
        Persistent Portal: 192.168.200.36:3260,1002
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.domain:lnx200.202
                Iface IPaddress: 192.168.200.202
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 3
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 65536
                FirstBurstLength: 65536
                MaxBurstLength: 65536
                ImmediateData: Yes
                InitialR2T: No
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 7  State: running
                scsi7 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdf          State: running
                scsi7 Channel 00 Id 0 Lun: 1
                        Attached scsi disk sdg          State: running
                scsi7 Channel 00 Id 0 Lun: 2
                        Attached scsi disk sdh          State: running
                scsi7 Channel 00 Id 0 Lun: 3
                        Attached scsi disk sdi          State: running
Target: iqn.1992-08.com.netapp:sn.101180456
        Current Portal: 10.72.201.37:3260,1000
        Persistent Portal: 10.72.201.37:3260,1000
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.domain:lnx200.202
                Iface IPaddress: 10.72.200.202
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 2
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 65536
                FirstBurstLength: 65536
                MaxBurstLength: 65536
                ImmediateData: Yes
                InitialR2T: No
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 6  State: running
        Current Portal: 192.168.200.37:3260,1002
        Persistent Portal: 192.168.200.37:3260,1002
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.domain:lnx200.202
                Iface IPaddress: 192.168.200.202
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 4
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 65536
                FirstBurstLength: 65536
                MaxBurstLength: 65536
                ImmediateData: Yes
                InitialR2T: No
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 8  State: running
Comment 11 gowrav 2011-02-06 07:02:58 UTC 
The "/etc/init.d/iscsi status" output mentioned in Comment 10, can be manually got by executing the command "iscsiadm -m session -P 3".

I'm not sure if the "status" output is designed to print verbose output about iscsi session. Can the output be just reduced to print a single line stating if the iscsi daemon is running or stopped (as done in RHEL 5 series)?
Comment 12 Mike Christie 2011-02-14 19:46:03 UTC 
(In reply to comment #11)
> The "/etc/init.d/iscsi status" output mentioned in Comment 10, can be manually
> got by executing the command "iscsiadm -m session -P 3".
> 
> I'm not sure if the "status" output is designed to print verbose output about
> iscsi session. Can the output be just reduced to print a single line stating if
> the iscsi daemon is running or stopped (as done in RHEL 5 series)?

No. I guess more people wanted the verbose info.

If you wanted to know about iscsid status you can do

service iscsid status.
Comment 13 gowrav 2011-02-15 04:18:58 UTC 
> 
> No. I guess more people wanted the verbose info.
> 
> If you wanted to know about iscsid status you can do
> 
> service iscsid status.

Then, in that case we can close this bug as fixed.
Comment 15 Laura Bailey 2011-05-13 01:54:03 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Starting or stopping the iSCSI service while accessing the root partition directly through an iSCSI disk could cause iSCSI to become unresponsive and incorrect status information to be reported. Attempting to stop the iSCSI service in this circumstance now warns that iSCSI cannot be shut down while Root is on an iSCSI disk, and all statuses are reported correctly.
Comment 16 errata-xmlrpc 2011-05-19 14:14:53 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0733.html
Comment 17 saroj parida 2014-12-05 03:35:47 UTC 
After successful installation of iscsi-initiator-utils package in Red Hat Enterprise 6.0, the service is not started. I could not start the service in manually also. Could please guide "What is the issues?". I have tried below method for starting iscsi service.

1). service iscsi restart/start/reload
     service iscsid restart/start/reload

2). service iscsi force-start
     service iscsid force-start

3). chkconfig iscsi on
     chkconfig iscsid on

But still I could not start the service. Could you please replay?
Note You need to log in before you can comment on or make changes to this bug.