Bug 1141510

Summary: iscsi error with chap enabled during discovery
Product: [Retired] oVirt Reporter: Gianluca Cecchi <gianluca.cecchi>
Component: vdsmAssignee: Dan Kenigsberg <danken>
Status: CLOSED CURRENTRELEASE QA Contact: Gil Klein <gklein>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.5CC: amureini, bazulay, bugs, danken, ecohen, gianluca.cecchi, gklein, iheim, mgoldboi, nicolas, nsoffer, oourfali, rbalakri, smizrahi, yeylon
Target Milestone: ---Keywords: Regression
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-17 12:22:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1073943    
Attachments:
Description Flags
vdsm+engine logs
none
vdsm log with the iSCSI error
none
engine.log.gz none

Description Gianluca Cecchi 2014-09-14 08:48:19 UTC
Description of problem:
Strange workflow for discovery of iSCSI targets when chap enabled.
More than required arguments passed.

Version-Release number of selected component (if applicable):
vdsm-jsonrpc-4.16.4-0.el6.noarch

How reproducible:
always

Steps to Reproduce:
1. make discovery enabling chap and inserting username/password
2. no lun discovered
3.

Actual results:
no lun discovered

Expected results:
discovery of new luns presented

Additional info:

trying to configure iSCSI storage domain.

I have configured a CentOS 6.5 server as sw iSCSI target with chap authentication.
port 3260 is open for tcp connections.
I have an oVirt host that is CentOS 6.5 and when trying to discover targets I get in /var/log/messages

Sep 12 23:50:19 ovnode04 vdsm jsonrpc.JsonRpcServer ERROR Internal server error#012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/yajsonrpc/__init__.py", line 486, in _serveRequest#012    res = method(**params)#012  File "/usr/share/vdsm/rpc/Bridge.py", line 264, in _dynamicMethod#012    raise InvalidCall(fn, methodArgs, e)#012InvalidCall: Attempt to call function: <bound method ISCSIConnection.discoverSendTargets of <API.ISCSIConnection object at 0x7f0a48145150>> with arguments: (u'iscsiuser', u'iscsipwd') error: discoverSendTargets() takes exactly 1 argument (3 given)
Sep 12 23:50:22 ovnode04 vdsm jsonrpc.JsonRpcServer ERROR Internal server error#012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/yajsonrpc/__init__.py", line 486, in _serveRequest#012    res = method(**params)#012  File "/usr/share/vdsm/rpc/Bridge.py", line 264, in _dynamicMethod#012    raise InvalidCall(fn, methodArgs, e)#012InvalidCall: Attempt to call function: <bound method ISCSIConnection.discoverSendTargets of <API.ISCSIConnection object at 0x7f0a480e5ad0>> with arguments: (u'iscsiuser', u'iscsipwd') error: discoverSendTargets() takes exactly 1 argument (3 given)

screenshot that returns no new devices found is here:
https://drive.google.com/file/d/0BwoPbcrMv8mvbVlycnJCTWJtNXc/edit?usp=sharing

BTW: chap user and password input fields could be wider, so that the whole words input can seen on the screen. there is plenty of space... Also password field should not be in clear text as it is now

The approach to successfully connect the LUN seems to be:
1) make discovery but with chap unchecked
2) the target then shows up
see
https://drive.google.com/file/d/0BwoPbcrMv8mvb1pET3VNMWJuRUk/edit?usp=sharing
3) now check the box for chap authentication and select login all
you now get the luns
https://drive.google.com/file/d/0BwoPbcrMv8mveExRMzN1Z0RtR1k/edit?usp=sharing
4) then select the lun(s) desired and select OK

is this correct? In case I think it should be disabled the chap option during discovery phase...
otherwise I have not configured correctly my iscsi target perhaps.
ovirt host network ip is 10.10.1.61

tgtadm --lld iscsi --mode target --op show
Target 1: iqn.2014-07.local.localdomain:store1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 1
            Initiator: iqn.1994-05.com.redhat:5d9b31319a8e
            Connection: 0
                IP Address: 10.10.1.61
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: p_iscsi_store1_l
            SCSI SN: 66666a41
            Size: 214738 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: rdwr
            Backing store path: /dev/drbd/by-res/iscsiha
            Backing store flags:
    Account information:
        iscsiuser
    ACL information:
        10.10.1.61
        10.10.1.62
        10.10.1.63

Comment 1 Piotr Kliczewski 2014-09-15 08:07:38 UTC
Please provide vdsm.log.

Comment 2 Nir Soffer 2014-09-15 12:25:02 UTC
I would test the same with jsonrpc disabled, to make sure that this is related to jsonrpc. If this also do not work on xmlrpc, then it sounds like storage issue.

Comment 3 Allon Mureinik 2014-09-15 16:33:46 UTC
Reproduced locally - this indeed seems to be a json issue. 
With XMLRPC I got an error about the vdsm verb failing due to fabricated authentication detials.
With JSONRPC I got the above issue.

vdsm version:
[root@mayhem ~]# rpm -qa | grep vdsm
vdsm-xmlrpc-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-jsonrpc-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-python-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-yajsonrpc-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-cli-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-python-zombiereaper-4.16.0-291.git9d8dc2a.fc20.noarch
vdsm-4.16.0-291.git9d8dc2a.fc20.x86_64

Engine:
Built from source, using commit hash a6387a99ec9d497aec12b5b5d5a406826c723649

Comment 4 Allon Mureinik 2014-09-15 16:38:09 UTC
Created attachment 937670 [details]
vdsm+engine logs

Comment 5 Gianluca Cecchi 2014-09-15 21:58:42 UTC
Created attachment 937722 [details]
vdsm log with the iSCSI error

Here it is the requested log file for vdsm.

Comment 6 Gianluca Cecchi 2014-09-15 22:02:39 UTC
Created attachment 937723 [details]
engine.log.gz

Here it is the gzipped engine.log

Comment 7 Gianluca Cecchi 2014-09-15 22:03:57 UTC
My vdsm related versions:

[root@ovnode04 ~]# rpm -qa | grep vdsm
vdsm-xmlrpc-4.16.4-0.el6.noarch
vdsm-yajsonrpc-4.16.4-0.el6.noarch
vdsm-jsonrpc-4.16.4-0.el6.noarch
vdsm-python-zombiereaper-4.16.4-0.el6.noarch
vdsm-cli-4.16.4-0.el6.noarch
vdsm-python-4.16.4-0.el6.noarch
vdsm-4.16.4-0.el6.x86_64

Comment 8 Sandro Bonazzola 2014-10-17 12:22:59 UTC
oVirt 3.5 has been released and should include the fix for this issue.