Bug 1056948 - iscsid daemon not started when vdsm starts and iscsi storage not available
Summary: iscsid daemon not started when vdsm starts and iscsi storage not available
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.3.0
Hardware: Unspecified
OS: Linux
Target Milestone: ---
: 3.4.0
Assignee: Nir Soffer
QA Contact: Aharon Canan
Whiteboard: storage
Depends On:
Blocks: 1058785 rhev3.4beta 1142926
TreeView+ depends on / blocked
Reported: 2014-01-23 08:53 UTC by Nir Soffer
Modified: 2016-02-10 17:16 UTC (History)
12 users (show)

Fixed In Version: av1
Doc Type: Bug Fix
Doc Text:
The iscsi daemon is now started when VDSM starts, and iSCSI storage is now available when VDSM starts.
Clone Of:
: 1058785 (view as bug list)
Last Closed: 2014-06-09 13:28:02 UTC
oVirt Team: Storage
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:0504 0 normal SHIPPED_LIVE vdsm 3.4.0 bug fix and enhancement update 2014-06-09 17:21:35 UTC
oVirt gerrit 23601 0 None None None Never
oVirt gerrit 23770 0 None None None Never
oVirt gerrit 23778 0 None None None Never

Description Nir Soffer 2014-01-23 08:53:36 UTC
Description of problem:

If /etc/iscsi/iscsi.conf does not contains this option:

    iscsid.startup = /etc/rc.d/init.d/iscsid force-start

Then iscsid is not started by iscsiadm when vdsm try to connect to a storage server, and iscsi storage is not available.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Find a machine with the missing iscsi configuration, or comment this option in iscsi configuration.
2. Reboot the machine

Actual results:

iscsid is not started when vdsm starts and iscsi storage not available

Expected results:

iscsid is started when vdsm starts and iscsi storage available

Additional info:

vdsm is configuring iscsi nodes node.startup option to "manual", so these nodes will not be available unless vdsm is connecting to the node. With this setting, "service iscsid start" may not start the iscsid daemon, and to start the daemon you need to use the "force-start" command.

vdsm used to "force-start" iscsid since 2011, and stopped to do this since http://gerrit.ovirt.org/14630 was merged in May 2013. It seems that we did not detect the issuse since our systems happen to have the needed iscsi.startup configuration option.

This bug was detected on a user machine on day after RHEVM 3.3.0 was released. We don't know yet how common is the iscsi configuration that cause this error.

Comment 1 Nir Soffer 2014-01-23 09:00:22 UTC
This was detected on a user data center with two hosts. Both hosts had the same iscsi ocnfiguraion without the iscsi.startup option.

Comment 2 Nir Soffer 2014-01-23 09:06:10 UTC
Workaround - edit /etc/iscsi/iscsi.conf and add this option:

    iscsid.startup = /etc/rc.d/init.d/iscsid force-start

Comment 4 Petr Spacek 2014-01-23 09:25:56 UTC
Note: We never touched the iscsid.conf by hand, the file was installed along with RHEL 6.3 (and possibly modified by RHEV 3.x during hypervisor installation).

Relevant parts from yum history:
$ sudo yum history info iscsi-initiator-utils | grep -A1 -P '(iscsi-initiator-utils|Transaction ID|[^ ] time)'
Transaction ID : 264
Begin time     : Tue Dec 10 20:54:03 2013
End time       :            21:01:21 2013 (7 minutes)
    Updated     iscsi-initiator-utils-                  @rhel-x86_64-server-6
    Update                                   @rhel-x86_64-server-6
Transaction ID : 77
Begin time     : Mon Mar 11 15:07:07 2013
End time       :            15:07:13 2013 (6 seconds)
    Updated iscsi-initiator-utils- @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3
    Update                @rhel-x86_64-server-6
Transaction ID : 1
Begin time     : Mon Sep  3 15:12:03 2012
End time       :            15:16:59 2012 (296 seconds)
    Dep-Install iscsi-initiator-utils-                 @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3

You can see that the file was not touched from December (and the machine was rebooted after the last upgrade in December) - it worked with vdsm-4.10.2-27.0.el6ev.x86_64.

Comment 5 Nir Soffer 2014-01-23 19:00:07 UTC
Lee, how common is this iscsi configuration in the field?

Comment 7 Matthew Davis 2014-01-27 15:21:44 UTC
fwiw, the North American Commercial SA team run 2 labs for customer facing demos. And we try to keep configuration rather standard (out of the box). Both our labs were upgraded from 3.2 -> 3.3 and both use an iscsi data domain and both hit this bug.

I'd say ANY install with an iscsi data domain will suffer from this bug.

Comment 9 Aharon Canan 2014-02-19 13:36:26 UTC
verified using vdsm-4.14.3-0.el6.x86_64

disable iscsid.startup = /etc/rc.d/init.d/iscsid force-start in /etc/iscsi/iscsi.conf 
reboot the host
iscsid started

Comment 12 errata-xmlrpc 2014-06-09 13:28:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.