Bug 1163948

Description of problem:
Tried to create an iSCSI storage domain, by discovering a target and login to it. Login to the target fails everytime, I tried with several hosts and 2 different storage servers. 
The storage servers I'm using expose 4 targets while connecting to them. It seems like vdsm is unable to connect to more than 1 target. Olny one login succeeds

Version-Release number of selected component (if applicable):
rhev 3.5 vt10
vdsm-4.16.7.4-1.el6ev.x86_64
iscsi-initiator-utils-6.2.0.873-13.el6.x86_64

How reproducible:
Always, with several hosts and with 2 different storage servers (XtremIO and EMC-VNX-5300)

Steps to Reproduce:
1. Have a host connected to a storage server, written in it as an initiator and expose LUNs to it. Make sure the host is not logged in to the target:
[root@green-vdsa ~]# iscsiadm -m session
iscsiadm: No active sessions.
2. Try to create an iSCSI storage domain using the host: discover the target and try to login
3.

Actual results:
The host login successfully to one of the targets. Login to the rest fails
Login fails everytime on vdsm:

Thread-879::ERROR::2014-11-13 19:58:25,872::hsm::2433::Storage.HSM::(connectStorageServer) Could not connect to storageServer
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 2430, in connectStorageServer
    conObj.connect()
  File "/usr/share/vdsm/storage/storageServer.py", line 367, in connect
    iscsi.addIscsiNode(self._iface, self._target, self._cred)
  File "/usr/share/vdsm/storage/iscsi.py", line 164, in addIscsiNode
    iscsiadm.node_login(iface.name, portalStr, target.iqn)
  File "/usr/share/vdsm/storage/iscsiadm.py", line 295, in node_login
    raise IscsiNodeError(rc, out, err)
IscsiNodeError: (19, ['Logging in to [iface: default, target: iqn.2008-05.com.xtremio:001e675ba171, portal: 10.35.146.129,3260] (multiple)'], ['iscsiadm: Could not login to [ifa
ce: default, target: iqn.2008-05.com.xtremio:001e675ba171, portal: 10.35.146.129,3260].', 'iscsiadm: initiator reported error (19 - encountered non-retryable iSCSI login failure
)', 'iscsiadm: Could not log into all portals'])


On engine:

2014-11-13 20:21:53,649 ERROR [org.ovirt.engine.core.bll.storage.DiscoverSendTargetsQuery] (ajp-/127.0.0.1:8702-2) Query DiscoverSendTargetsQuery failed. Exception message is Vd
cBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed in vdscommand to DiscoverSendTargetsVDS, error = Faile
d discovery of iSCSI targets: "portal=IscsiPortal(hostname=u'10.35.160.129', port=3260), err=(21, [], ['iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'is
csiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 1
0.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'i
scsiadm: connection login retries (reopen_max) 5 exceeded', 'iscsiadm: No portals found'])" (Failed with error iSCSIDiscoveryError and code 475) : org.ovirt.engine.core.common.e
rrors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed in vdscommand to DiscoverSend
TargetsVDS, error = Failed discovery of iSCSI targets: "portal=IscsiPortal(hostname=u'10.35.160.129', port=3260), err=(21, [], ['iscsiadm: cannot make connection to 10.35.160.12
9: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: ca
nnot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.1
29: No route to host', 'iscsiadm: connection login retries (reopen_max) 5 exceeded', 'iscsiadm: No portals found'])" (Failed with error iSCSIDiscoveryError and code 475): org.ov
irt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed in vd
scommand to DiscoverSendTargetsVDS, error = Failed discovery of iSCSI targets: "portal=IscsiPortal(hostname=u'10.35.160.129', port=3260), err=(21, [], ['iscsiadm: cannot make co
nnection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route
 to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make connection to 10.35.160.129: No route to host', 'iscsiadm: cannot make c
onnection to 10.35.160.129: No route to host', 'iscsiadm: connection login retries (reopen_max) 5 exceeded', 'iscsiadm: No portals found'])" (Failed with error iSCSIDiscoveryErr
or and code 475)
        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.DiscoverSendTargetsQuery.executeQueryCommand(DiscoverSendTargetsQuery.java:16) [bll.jar:]
        at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:73) [bll.jar:]

It seems like the host is unable to connect to more than 1 target

Expected results:
Login to all exposed targets should succed. Seems like a regression in vdsm

Additional info:
engine.log 
logs from host