Bug 598941 - iscsi_tcp: datalen 7041824 > 26214 ; connection errors
Summary: iscsi_tcp: datalen 7041824 > 26214 ; connection errors
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: iscsi-initiator-utils
Version: 5.5
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Mike Christie
QA Contact: Red Hat Kernel QE team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-06-02 11:31 UTC by David Kovalsky
Modified: 2014-03-31 23:45 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-11-23 15:51:20 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
dump of node config (1.54 KB, application/octet-stream)
2010-06-02 11:31 UTC, David Kovalsky
no flags Details

Description David Kovalsky 2010-06-02 11:31:45 UTC
Created attachment 418996 [details]
dump of node config

I have Xen Dom0 setup, which mounts iscsi device as sde. /dev/sde serves as a disk for a paravirt guest. 

I'm able to copy data to the device without any issues. I 'dd-ed' one of the partitions without any errrors.

But when attemting to start a VM, the boot fails a /var/log/messages reveals these errors:

kernel:  connection2:0: iscsi_tcp: datalen 7041824 > 262144
kernel:  connection2:0: detected conn error (1006)
iscsid: connection2:0 is operational after recovery (1 attempts)

or

iscsid: Kernel reported iSCSI connection 1:0 error (1006) state (3)
kernel:  connection1:0: received itt 0 expected session age (e)
kernel:  connection1:0: detected conn error (1010)
iscsid: connection1:0 is operational after recovery (1 attempts)
iscsid: Kernel reported iSCSI connection 1:0 error (1010) state (3)
iscsid: connection1:0 is operational after recovery (1 attempts)
...


kernel-xen-2.6.18-194.3.1.el5
iscsi-initiator-utils-6.2.0.871-0.16.el5

I haven't made any changes to the config file. Not using auth.

dump from /var/lib/iscsi/nodes/${mynode}/default attached


If it matters, target is scsi-target-utils-0.0-6.20091205snap.el5_5.2
1GbE devices have MTU set to 9000. 

I'm don't know what info might be relavant for you, so please ask if I've missed anything important. I have the setup handy, so I can provide more logs if necessary.

Comment 1 Mike Christie 2010-06-02 17:14:00 UTC
Hi,

This:
kernel:  connection2:0: iscsi_tcp: datalen 7041824 > 262144
indicates that the target and initiator negotiated that the target would only send up to 262144 bytes to the initiator in a iscsi packet. However, for some reason either the target went bonkers and sent us a lot more data than it should or the initiator messed up and is mis-reading the length or the packet got corrupted on its way to the initiator.


On the target or initiator would be it be possible to take a wireshark or tcpdump trace?  You can just run them with the default options to get iscsi data.

Comment 2 David Kovalsky 2010-06-07 22:08:46 UTC
So I tried to setup wireshark with the same setup I had, but I'm stuck on bug 601420.

Comment 4 Mike Christie 2010-07-30 15:40:30 UTC
Hey David,

Are you using something like xvd with iscsi?

Comment 5 David Kovalsky 2010-08-04 15:21:20 UTC
Hi Mike, 

doesn't seem so - I grepped both the client & the server for xvd and found no match. How can I tell? 

(sorry, I don't have much experience with iSCSI and Google is not helping me much today)

Comment 6 David Kovalsky 2010-08-04 15:28:25 UTC
More though - if XVD means Xen Virtual Disks, then the answer is yes:

The machine serving is running normal kernel (with KVM), the machine connecting is running the xen kernel. Config file for paravirt machines has:
...
disk = [ "phy:/dev/sde,xvda,w", ]
...

Comment 7 Mike Christie 2010-08-04 17:39:12 UTC
(In reply to comment #6)
> More though - if XVD means Xen Virtual Disks, then the answer is yes:
> 

Yeah, sorry for the confusion.

> The machine serving is running normal kernel (with KVM), the machine connecting
> is running the xen kernel. Config file for paravirt machines has:
> ...
> disk = [ "phy:/dev/sde,xvda,w", ]
> ...    

Were you the one that gave me access to the box/vm that was causing trouble? If so I cannot find the email with the login info. Could you resend it?

Comment 9 RHEL Program Management 2010-08-09 19:51:51 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.


Note You need to log in before you can comment on or make changes to this bug.