Bug 747479 - iscsi targets starting before network is up for non-critical mounts on boot
iscsi targets starting before network is up for non-critical mounts on boot
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: iscsi-initiator-utils (Show other bugs)
16
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Mike Christie
Fedora Extras Quality Assurance
AcceptedNTH
:
Depends On:
Blocks: F16-accepted/F16FinalFreezeExcept
  Show dependency treegraph
 
Reported: 2011-10-19 21:35 EDT by Tim Flink
Modified: 2012-06-26 12:08 EDT (History)
5 users (show)

See Also:
Fixed In Version: iscsi-initiator-utils-6.2.0.872-16.fc16
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-02-07 21:28:19 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
boot log excerpt showing iscsi starting before network and failing (3.31 KB, text/plain)
2011-10-19 21:41 EDT, Tim Flink
no flags Details

  None (edit)
Description Tim Flink 2011-10-19 21:35:43 EDT
Description of problem:

I did a minimal install (no NM) for F16 i386 and added /opt as a 20G
iSCSI target.

When I boot, iscsi and iscsid attempt to start before networking is up
and the target is not mounted.

Version-Release number of selected component (if applicable):
iscsi-initiator-utils-6.2.0.072-12.fc16

How reproducible:
Every time

Steps to Reproduce:
1. install F16 with an iSCSI target as /opt
2. reboot
  
Actual results:
/opt is not mounted on reboot because the network isn't up before the system tries to mount the iSCSI target

Expected results:
/opt is mounted at boot

Additional info:

The only way I've been able to get the iscsi target to mount @ boot is to change either the iscsid or iscsi init script to depend on $network.
Comment 1 Tim Flink 2011-10-19 21:41:34 EDT
Created attachment 529149 [details]
boot log excerpt showing iscsi starting before network and failing
Comment 2 Tim Flink 2011-10-20 11:10:10 EDT
Proposing as NTH for Fedora 16 final as it violates the following final release criterion [1]:

The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel).

However, it only affects systems without NetworkManager and doesn't seem appropriate for a blocker.

[1] http://fedoraproject.org/wiki/Fedora_16_Final_Release_Criteria
Comment 3 Adam Williamson 2011-10-28 15:22:24 EDT
This was accepted at the 2011-10-21 NTH review meeting but not updated, updating now.
Comment 4 Nathan G. Grennan 2011-11-08 14:35:32 EST
I am seeing this same issue.
Comment 5 Mike Christie 2011-12-06 17:18:08 EST
Nathan,

I was going to just merge a fix so that network is added to the RequiredStart section of the iscsi init script, but while testing this I noticed it does not help in F16 with the rpms all up to date (I thought this fixed the problem before).

In your setups, do you have network init script setup? Is it linked properly in /etc/rc*.d/ ? Does

chkconfig --list network

show it is setup properly?

If you run

chkconfig network on

does it then work? Is the problem that we want this to be done automatically?

Or in your setups is the network coming up after the iscsi script? If so, could you send the output of

ls -l /etc/rc5.d/*

(replace 5 with your value).
Comment 6 Nathan G. Grennan 2011-12-24 13:05:10 EST
chkconfig --list network

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

network         0:off   1:off   2:on    3:on    4:on    5:on    6:off





lrwxrwxrwx  1 root root 16 Dec  6  2009 /etc/rc5.d/S07iscsid -> ../init.d/iscsid
lrwxrwxrwx  1 root root 16 Jan 23  2010 /etc/rc5.d/S08nvidia -> ../init.d/nvidia
lrwxrwxrwx  1 root root 14 Sep 21  2010 /etc/rc5.d/S09capi -> ../init.d/capi
lrwxrwxrwx  1 root root 14 Sep 21  2010 /etc/rc5.d/S09isdn -> ../init.d/isdn
lrwxrwxrwx  1 root root 17 Nov  8 11:08 /etc/rc5.d/S10network -> ../init.d/network
lrwxrwxrwx  1 root root 15 Dec 20 16:17 /etc/rc5.d/S13iscsi -> ../init.d/iscsi
Comment 7 Adam Williamson 2012-01-23 15:03:01 EST
Ping, where are we on this? Any chance of a fix for F17?



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers
Comment 8 Nathan G. Grennan 2012-02-03 19:13:22 EST
I found the cause of the bug for me. I use network instead of network-manager, and /var/lock/subsys/network doesn't exist yet, even though the interfaces are up. When I comment out the check, it works.


    # if the network isn't up yet exit cleanly, NetworkManager will call us
    # again when the network is up
    [ ! -f /var/lock/subsys/network ] && ! nm-online -x >/dev/null 2>&1 && exit 3
Comment 9 Nathan G. Grennan 2012-02-03 19:22:54 EST
I just double checked things and found I am wrong on the details, but commenting out the line does fix it for me.

Here is the timing:
Feb  3 16:14:04 proton network[1299]: Bringing up loopback interface:  [  OK  ]
Feb  3 16:14:06 proton network[1299]: Bringing up interface eth0:  [  OK  ]
Feb  3 16:14:08 proton network[1299]: Bringing up interface eth1:  [  OK  ]
Feb  3 16:14:11 proton network[1299]: Bringing up interface eth2:  [  OK  ]
Feb  3 16:14:22 proton iscsi[1359]: Starting iscsi: [  OK  ]

ls -al --time-style full-iso /var/lock/subsys/network
-rw-r--r-- 1 root root 0 2012-02-03 16:14:11.153393551 -0800 /var/lock/subsys/network

So /var/lock/subsys/network does get created right after eth2 comes up, and iscsi comes up 11 seconds later.

I also proved it isn't the line below during testing by changing the exit code for the network line from 3 to 4.

[ $? -eq 0 ] || exit 3


So it all comes down to the line below not working.

[ ! -f /var/lock/subsys/network ] && ! nm-online -x >/dev/null 2>&1 && exit
3
Comment 10 Mike Christie 2012-02-04 04:19:30 EST
This should be fixed in
iscsi-initiator-utils-6.2.0.872-16.fc17/iscsi-initiator-utils-6.2.0.872-15.fc16.
Comment 11 Fedora Update System 2012-02-04 04:33:31 EST
iscsi-initiator-utils-6.2.0.872-15.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/iscsi-initiator-utils-6.2.0.872-15.fc16
Comment 12 Fedora Update System 2012-02-04 18:52:21 EST
Package iscsi-initiator-utils-6.2.0.872-15.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing iscsi-initiator-utils-6.2.0.872-15.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-1333/iscsi-initiator-utils-6.2.0.872-15.fc16
then log in and leave karma (feedback).
Comment 13 Nathan G. Grennan 2012-02-04 21:54:42 EST
Works for me. Thank you!
Comment 14 Fedora Update System 2012-02-07 21:28:19 EST
iscsi-initiator-utils-6.2.0.872-16.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.